Polycrystalline CeVO3 has been studied by neutron powder diffraction (NPD), specific heat, and magnetization measurements. CeVO3 becomes magnetically ordered below TN≈136 K and an anomalous-diamagnetism effect is observed below Tt≈124 K under zero-field-cooling conditions. Simultaneously, a crystallographic transition is observed by NPD at Tt, from the high-temperature orthorhombic structure, space group Pnma, to a low-temperature monoclinic structure, space group P21/n. When cooling the sample across this transition the perovskite distortion evolves from the O type, with b/(c√2)>1, to the O′ type, characterized by b/(c√2)<1, involving a significant increase in the distortion of the VO6 octahedra. Neutron diffraction measurements revealed that, below TN, the magnetic ordering is characterized by the propagation vector k=0, concerning only the vanadium magnetic moments, which adopt a spin arrangement given by the basis vector (0,0,Gz). Thus, the magnetic moments are antiferromagnetically coupled in the a-c plane, and the a-c layers couple ferromagnetically along the b direction. At around 50 K there is a transition to a different magnetic spin arrangement, also characterized by k=0. A ferromagnetic component appears on the V3+ cations, which adopt the magnetic structure (Fx,0,Gz); on the other hand, the cerium magnetic moments also become ordered with a spin arrangement given by (Fz′,0,Gz′). At T=2.6 K the magnetic moments for the V3+ and Ce3+ ions are 1.72(4)μB and 0.46(3)μB, respectively.