ABSTRACT We study the Intra-Halo Stellar Component (IHSC) of Milky Way-mass systems up to galaxy clusters in the Horizon-AGN cosmological hydrodynamical simulation. We identify the IHSC using an improved phase-space galaxy finder algorithm which provides an adaptive, physically motivated, and shape-independent definition of this stellar component, that can be applied to haloes of arbitrary masses. We explore the IHSC mass fraction – total halo’s stellar mass, $f_{M_{*,\mathrm{IHSC}}} - M_{*}$, relation, and the physical drivers of its scatter. We find that on average, the $f_{M_{*,\mathrm{IHSC}}}$ increases with total stellar mass, with the scatter decreasing strongly with mass from 2 dex at $M_{*,\mathrm{tot}}≃ 10^{11}\, \mathrm{M}_⊙$ to 0.3 dex at group masses. At high masses, $M_{*,\mathrm{tot}}\gt 10^{11.5}\, \mathrm{M}_⊙$, $f_{M_{*,\mathrm{IHSC}}}$ increases with the number of substructures, and with the mass ratio between the central galaxy and largest satellite, at fixed M*, tot. From mid-size groups and systems below $M_{*,\mathrm{tot}}\lt 10^{12}\, \mathrm{M}_⊙$, we find that the central galaxy’s stellar rotation-to-dispersion velocity ratio, V/σ, displays the strongest (anti)-correlation with $f_{M_{*,\mathrm{IHSC}}}$ at fixed M*, tot of all the galaxy and halo properties explored, transitioning from $f_{M_{*,\mathrm{IHSC}}}\lt 0.1$ per cent for high V/σ, to $f_{M_{*,\mathrm{IHSC}}}≈ 5$ per cent for low V/σ galaxies. By studying the $f_{M_{*,\mathrm{IHSC}}}$ temporal evolution, we find that, in the former, mergers not always take place, but if they did, they happened early (z > 1), while the high $f_{M_{*,\mathrm{IHSC}}}$ population displays a much more active merger history. In the case of massive groups and galaxy clusters, $M_{*,\mathrm{tot}}\gtrsim 10^{12}\, \mathrm{M}_⊙$, a fraction $f_{M_{*,\mathrm{IHSC}}}≈ 10-20$ per cent is reached at z ≈ 1 and then they evolve across lines of constant $f_{M_{*,\mathrm{IHSC}}}$ modulo some small perturbations. Because of the limited simulation’s volume, the latter is only tentative and requires a larger sample of simulated galaxy clusters to confirm.