High-resolution two-photon photoemission spectroscopy was used to investigate the energy shift and lifetime broadening of the first image state on Pd(111) under adsorption of CO, O2 and H2 at room temperature and at 90 K. For all adsorbates the binding energy increases with coverage, which clearly shows that the image state is sensitive not only to the shift of the vacuum level, but also to the variation of the potential in the vicinity of the surface caused by the adsorbate. The intrinsic linewidth of the image state increases under adsorption and implies a reduced lifetime of the electron in the image state. It can be explained in terms of elastic scattering of the image state at confining adcenters and inelastic scattering by these adcenters into bulk states.