The magnetism of Co-Rh nanoparticles is investigated experimentally and theoretically. The particles (approximately 2 nm) have been synthesized by decomposition of organometallic precursors in mild conditions of pressure and temperature, under hydrogen atmosphere and in the presence of a polymer matrix. The magnetic properties are determined by SQUID, Mössbauer spectroscopy, and X-ray magnetic circular dichroism (XMCD). The structural and chemical properties are characterized by wide angle X-ray scattering, transmission electronic microscopy and X-ray absorption near edge spectroscopy. All the studied Co-Rh clusters are magnetic with an average spin moment per atom mu that is larger than the one of macroscopic crystals or alloys with similar concentrations. The experimental results and comparison with theory suggest that the most likely chemical arrangement is a Rh core, with a Co-rich outer shell showing significant Co-Rh mixing at the interface. Measured and calculated magnetic anisotropy energies (MAEs) are found to be higher than in pure Co clusters. Moreover, one observes that the MAEs can be tuned to some extent by varying the Rh concentration. These trends are well accounted for by theory, which in addition reveals important spin and orbital moments induced at the Rh atoms as well as significant orbital moments at the Co atoms. These play a central role in the interpretation of experimental data as a function of Co-Rh content. A more detailed analysis from a local perspective shows that the orbital and spin moments at the Co-Rh interface are largely responsible for the enhancement of the magnetic moments and magnetic anisotropy.