The ability to produce ultrashort, high-brightness X-ray pulses is revolutionizing the field of ultrafast X-ray spectroscopy. Free-electron laser (FEL) facilities are driving this revolution, but unique aspects of the FEL process make the required characterization and use of the pulses challenging. In this paper, we describe a number of developments in the generation of ultrashort X-ray FEL pulses, and the concomitant progress in the experimental capabilities necessary for their characterization and use at the Linac Coherent Light Source. This includes the development of sub-femtosecond hard and soft X-ray pulses, along with ultrafast characterization techniques for these pulses. We also describe improved techniques for optical cross-correlation as needed to address the persistent challenge of external optical laser synchronization with these ultrashort X-ray pulses. This article is part of the theme issue ‘Measurement of ultrafast electronic and structural dynamics with X-rays’.