Abstract The delay time distribution of neutron star mergers provides critical insights into binary evolution processes and the merger rate evolution of compact object binaries. However, current observational constraints on this delay time distribution rely on the small sample of Galactic double neutron stars (with uncertain selection effects), a single multimessenger gravitational wave event, and indirect evidence of neutron star mergers based on r-process enrichment. We use a sample of 68 host galaxies of short gamma-ray bursts to place novel constraints on the delay time distribution and leverage this result to infer the merger rate evolution of compact object binaries containing neutron stars. We recover a power-law slope of α = − 1.83 − 0.39 + 0.35 (median and 90% credible interval) with α < −1.31 at 99% credibility, a minimum delay time of t min = 184 − 79 + 67 Myr with t min > 72 Myr at 99% credibility, and a maximum delay time constrained to t max > 7.95 Gyr at 99% credibility. We find these constraints to be broadly consistent with theoretical expectations, although our recovered power-law slope is substantially steeper than the conventional value of α = −1, and our minimum delay time is larger than the typically assumed value of 10 Myr. Pairing this cosmological probe of the fate of compact object binary systems with the Galactic population of double neutron stars will be crucial for understanding the unique selection effects governing both of these populations. In addition to probing a significantly larger redshift regime of neutron star mergers than possible with current gravitational wave detectors, complementing our results with future multimessenger gravitational wave events will also help determine if short gamma-ray bursts ubiquitously result from compact object binary mergers.