The scattering of inertia-gravity waves by large-scale geostrophic turbulence in a rapidly rotating, strongly stratified fluid leads to the diffusion of wave energy on the constant-frequency cone in wavenumber space. We derive the corresponding diffusion equation and relate its diffusivity to the wave characteristics and the energy spectrum of the turbulent flow. We check the predictions of this equation against numerical simulations of the three-dimensional Boussinesq equations in initial-value and forced scenarios with horizontally isotropic wave and flow fields. In the forced case, wavenumber diffusion results in a $k^{-2}$ wave energy spectrum consistent with as-yet-unexplained features of observed atmospheric and oceanic spectra.