To fully exploit ultra-short X-ray pulse durations routinely available at X-ray free-electron lasers to follow out-of-equilibrium dynamics, inherent arrival time fluctuations of the X-ray pulse with an external perturbing laser pulse need to be measured. In this work, two methods of arrival time measurement were compared to measure the arrival time jitter of hard X-ray pulses. The methods were photoelectron streaking by a THz field and a transient refractive index change of a semiconductor. The methods were validated by shot-to-shot correction of a pump–probe transient reflectivity measurement. An ultimate shot-to-shot full width at half-maximum error between the devices of 19.2 ± 0.1 fs was measured.