We use element-resolved IR-pump/extreme ultraviolet-probe experiments to disentangle the ultrafast interplay of the magnetic sublattices of an ordered crystalline magnetic alloy. As a paradigmatic example, we investigate the case of the FeRh alloy, which shows a delayed response for the different components. Furthermore, a detailed time-resolved magneto-optic study shows that the data can be analyzed by only assuming Elliot-Yafet-like scattering, as the underlying mechanism for ultrafast demagnetization, resulting in an unexpected nonmonotonic dependence of the spin-flip rate, as a function of quenching.