Neutron reflectometry and isotope multilayers were used to investigate self-diffusion in covalent amorphous solids during isothermal annealing and its correlation to structural relaxation. Amorphous silicon nitride was chosen as a model system. Neutron reflectometry is a superior method to measure very low self-diffusivities, occurring in covalent solids, by applying only short time anneals. This allows one to determine time dependent changes of diffusivity over a broad temperature range before crystallization of the metastable solid occurs. The measured nitrogen diffusivities decrease by more than one order of magnitude during annealing between 950 and 1150 °C, while at the same time also a decrease of the atomic density is observed. We interpret this behavior as a structural relaxation of the amorphous network structure that is governed by annihilation of interstitial-like defects.