ABSTRACT Absorbers in the spectrum of background objects probe the circumgalactic medium (CGM) surrounding galaxies, but its physical properties remain unconstrained. We use the cosmological hydrodynamical simulation TNG50 to statistically trace the origins of ${\rm H\, {\small I}}$ Ly α absorbers around galaxies at z = 0.5 with stellar masses ranging from 108 to 1011 M⊙. We emulate observational CGM studies by considering all gas within a line of sight velocity range of ±500 kms−1 from the central, to quantitatively assess the impact of other galaxy haloes and overdense gas in the IGM that intersect sightlines. We find that 75 per cent of ${\rm H\, {\small I}}$ absorbers with column densities $\log [N(\mbox{${\rm H\, {\small I}}$})/\rm {cm}^{-2}]\gt 16.0$ trace the central galaxy within ±150 (80) kms−1 of M* = 1010(108) M⊙ central galaxies. The impact of satellites to the total absorber fraction is most significant at impact parameters 0.5Rvir < b < Rvir, and satellites with masses below typical detection limits (M* < 108 M⊙) account for 10 (40) per cent of absorbers that intersect any satellite bound to 1010 and 1011 (109) M⊙ centrals. After confirming outflows are more dominant along the minor axis, we additionally show that at least 20 per cent of absorbers exhibit no significant radial movement, indicating that absorbers can also trace quasi-static gas. Our work shows that determining the stellar mass of galaxies at zabs is essential to constrain the physical origin of the gas traced in absorption, which in turn is key to characterizing the kinematics and distribution of gas and metals in the CGM.