Surface damaging processes of crystalline silicon and alumina (α-Al2O3) by gas ion (H+, He+, Ar+, 0.5–5 keV) irradiation at a moderate temperature range (300–450 K) were examined by reflection high-energy electron diffraction (RHEED). In order to explain the changes in RHEED intensities, a phenomenological kinetic equation was extended to take a dynamical interaction between hydrogen and target atoms into account. Apparent activation energies for simultaneous recovery and the cross-sections for damage production were derived and compared for different implant species by fitting the kinetic equation to the experimental data. The obtained physical parameters showed that the surface damage by inert gas irradiation could be understood simply by the dpa (displacement per atom) rates due to nuclear stopping. In case of H+ irradiation, on the other hand, electronic stopping and chemical interaction between hydrogen and the matrix atoms significantly affected the damaging rate and thermal recovery. In particular, hydrogen released from a weakly bound state in the Al2O3 matrix suppressed the thermal recovery at the moderate temperatures.