We present a theoretical investigation of the elastic Rayleigh scattering of X-rays by atomic targets. Special attention is paid to the question of how the polarization of the scattered photons is affected if the incident light is itself polarized. In particular, we found that the circular polarization of the incoming X-rays may lead to a remarkable modification of the linear polarization of the scattered photons. Based on this ’circular-to-linear-polarization-transfer’ and on the fact that the linear polarization of X-rays can be conveniently observed by solid-state Compton detectors, we argue that Rayleigh scattering may be used as a tool for circular polarimetry of hard X-rays. To illustrate our proposal, we performed detailed calculations of 145 and 500 keV circularly polarized photons scattered by lead atoms. Based on these calculations, we found that the photon scattering under large angles with respect to the incident beam direction is most favorable for the circular polarimetry of hard X-rays. In particular, for 500 keV photon energy and scattering angles around 70 deg we found a remarkable modification of the linear polarization of scattered light for the case when the incident radiation is circularly polarized.