We present a cross-correlation analysis to constrain the faint galaxy contribution to the cosmic X-ray background (XRB). Cross-correlating faint optical galaxy catalogues with unidentified X-ray sources from 3 deep ROSAT fields we find that B < 23 galaxies account for 20+-7 % of all X-ray sources to a flux limit of S(0.5-2.0 keV)=4x10^{-15} erg s-1 cm-2. To probe deeper, galaxies are then cross-correlated with the remaining unresolved X-ray images. A highly significant signal is obtained on each field. Allowing for the effect of the ROSAT point-spread function, and deconvolving the effect of galaxy clustering, we find that faint B < 23 galaxies directly account for 23+-3% of the unresolved XRB at 1keV. Using the optical magnitude of faint galaxies as probes of their redshift distribution, we find evidence for strong evolution in their X-ray luminosity, parameterised with the form L_x ∝ (1+z)^3.2+-1.0. Extrapolation to z=2 will account for 40+-10% of the total XRB at 1keV. The nature of the emitting mechanism in these galaxies remains unclear, but we argue that obscured and/or low luminosity AGN provide the most natural explanation.