It is demonstrated that the temporal evolution of a spin-wave induced inverse spin Hall effect voltage in a magnetic insulator-nonmagnetic metal structure is distinctly different from that of the directly excited (microwave pulse driven) spin-wave mode from which it originates. The differences in temporal behavior provide compelling evidence that incoherent secondary spin-wave modes, having a range of different characteristic lifetimes, make an important contribution to spin pumping at the insulator-metal interface. © 2011 American Institute of Physics.