We present a combined experimental and theoretical study on the rotationally inelastic scattering of OH ($X\,^2Π_{3/2}, J=3/2, f$) radicals with the collision partners He, Ne, Ar, Kr, Xe, and D$_2$ as a function of the collision energy between $∼ 70$ cm$^{-1}$ and 400~cm$^{-1}$. The OH radicals are state selected and velocity tuned prior to the collision using a Stark decelerator, and field-free parity-resolved state-to-state inelastic relative scattering cross sections are measured in a crossed molecular beam configuration. For all OH-rare gas atom systems excellent agreement is obtained with the cross sections predicted by close-coupling scattering calculations based on accurate \emph{ab initio} potential energy surfaces. This series of experiments complements recent studies on the scattering of OH radicals with Xe [Gilijamse \emph{et al.}, Science {\bf 313}, 1617 (2006)], Ar [Scharfenberg \emph{et al.}, Phys. Chem. Chem. Phys. {\bf 12}, 10660 (2010)], He, and D$_2$ [Kirste \emph{et al.}, Phys. Rev. A {\bf 82}, 042717 (2010)]. A comparison of the relative scattering cross sections for this set of collision partners reveals interesting trends in the scattering behavior. ; Comment: 10 pages, 5 figures