We report the successful synthesis of single-phase polycrystalline CeFeO3, and a thorough study of its magnetic and thermal properties. An abrupt spin reorientation transition occurs at T = 240 K, indicating a first-order phase transition from Γ4 (Gx, Ay, Fz) to Γ1 (Ax, Gy, Cz). A distinct thermal anomaly in the specific heat due to spin reorientation has been detected, with the onset and completion of the spin reorientation at 240 K and 220 K, respectively. The low-temperature specific heat exhibits a Schottky anomaly caused by paramagnetic Ce3+. The very-low-temperature specific heat increases with increasing the applied magnetic field, suggesting an antiferromagnetic transition of Ce3+ exists below T = 1.8 K.