CaV2O4 is a spin-1 antiferromagnet, where the magnetic vanadium ions have an orbital degree of freedom and are arranged on quasi-one-dimensional zigzag chains. The first- and second-neighbor vanadium separations are approximately equal suggesting frustrated antiferromagnetic exchange interactions. High-temperature susceptibility and single-crystal neutron-diffraction measurements are used to deduce the dominant exchange paths and orbital configurations. The results suggest that at high temperatures CaV2O4 behaves as a Haldane chain, but at low temperatures, it is a spin-1 ladder. These two magnetic structures are explained by different orbital configurations and show how orbital ordering can drive a system from one exotic spin Hamiltonian to another.