We investigate the magnetic properties of three Mn6 single molecule magnets by means of inelastic neutron scattering and frequency domain magnetic resonance spectroscopy. The experimental data reveal that small structural distortions of the molecular geometry produce a significant effect on the energy level diagram and therefore on the magnetic properties of the molecule. We show that the giant spin model completely fails to describe the spin level structure of the ground spin multiplets. We analyze theoretically the spin Hamiltonian for the low spin Mn6 molecule S 4 and we show that the excited S multiplets play a key role in determining the effective energy barrier for the magnetization reversal, in analogy to what was previously found for the two high spin Mn6 S 12 molecules S. Carretta et al., Phys. Rev. Lett. 100, 157203 2008