Orthorhombic α-BaTeMoO nonlinear optical single crystals were investigated at ambient pressure by micro-Raman and infrared spectroscopy with a focus on the polarization properties of the vibrational modes. These results were analyzed based on classical lattice dynamics calculations, allowing us to propose the normal-mode symmetries and assignments. In addition to the ambient-pressure studies, high-pressure Raman scattering studies were performed. These studies showed the onset of a reversible first-order phase transition near 3.5 GPa. The pressure dependence of Raman bands provides strong evidence that the phase transition involves significant distortion of the TeO (x = 3,4) polyhedra, whereas the MoO octahedra are less affected. A large increase in the number of observed bands points to lower symmetry of the high-pressure phase.