The availability of ultrafast pulses of coherent hard x-rays from the Linac Coherent Light Source opens new opportunities for studies of atomic-scale dynamics in amorphous materials. Here we show that single ultrafast coherent x-ray pulses can be used to observe the speckle contrast in the high-angle diffraction from liquid Ga and glassy Ni2Pd2P and B2O3. We determine the thresholds above which the x-ray pulses disturb the atomic arrangements. Furthermore, high contrast speckle is observed in scattering patterns from the glasses integrated over many pulses, demonstrating that the source and optics are sufficiently stable for x-ray photon correlation spectroscopy studies of dynamics over a wide range of time scales.