The recently demonstrated chiral modes of lattice motion carry angular momentum and therefore directly couple to magnetic fields. Notably, their magnetic moments are predicted to be strongly influenced by electronic contributions. Here, we have studied the magnetic response of transverse optical phonons in a set of Pb _{1− x} Sn _x Te films, which is a topological crystalline insulator for x > 0.32 and has a ferroelectric transition at an x -dependent critical temperature. Polarization-dependent terahertz magnetospectroscopy measurements revealed Zeeman splittings and diamagnetic shifts, demonstrating a large phonon magnetic moment. Films in the topological phase exhibited phonon magnetic moment values that were larger than those in the topologically trivial samples by two orders of magnitude. Furthermore, the sign of the effective phonon g -factor was opposite in the two phases, a signature of the topological transition according to our model. These results strongly indicate the existence of interplay between the magnetic properties of chiral phonons and the topology of the electronic band structure.