Significance Our study links diversity dynamics of fossil large mammals through time to primary productivity, i.e. net production of plant biomass. Spatial diversity patterns of extant terrestrial animals are often correlated with present-day primary productivity, but it is unclear whether the relationship holds throughout the geological past. Here we show that higher primary productivity was consistently associated with higher mammalian diversity throughout the geological period of the Neogene, supporting the hypothesis that energy flow from plants to consumers is a key factor determining the level of biodiversity. Our comparison of the fossil diversity–productivity relationship with present-day data suggests that human activity and Pleistocene climate change have conspired to dissolve the relationship that has characterized our planet over 20 My.