Inelastic neutron scattering was used to measure phonon spectra in MgB2 and Mg0.75Al0.25B2 from 7 to 750 K to investigate anharmonicity and adiabatic electron-phonon coupling. First-principles calculations of phonons with a linear response method were performed at multiple unit cell volumes, and the Helmholtz free energy was minimized to obtain the lattice parameters and phonon dynamics at elevated temperature in the quasiharmonic approximation. Most of the temperature dependence of the phonon density of states could be understood with the quasiharmonic approximation, although there was also significant thermal broadening of the phonon spectra. In comparison to Mg0.75Al0.25B2, in the energy range of 60 to 80 meV the experimental phonon spectra from MgB2 showed a nonmonotonic change with temperature around 500 K. This may originate from a change with temperature of the adiabatic electron-phonon coupling.