An inelastic neutron scattering experiment has been performed in order to determine the crystal field potential acting at the rare earth site in the paramagnetic phase of ErCu4Al8 and ErFe4Al8 intermetallic compounds. The profile refinement of the spectra measured at different temperatures, and experimental information on the electric field gradient available in the literature, allow a reliable estimate of the parameters for the tetragonal crystal field. Positive values of the lowest order term B20 suggest an easy plane magnetic anisotropy of the Er sublattice, but the higher order terms could play an important role in determining the magnetisation direction.