Polar phonons of HgCr2S4 and CdCr2S4 are studied by far-infrared spectroscopy as a function of temperature and external magnetic field. Eigenfrequencies, damping constants, effective plasma frequencies, Lyddane-Sachs-Teller relations, and effective charges are determined. Ferromagnetic CdCr2S4 and antiferromagnetic HgCr2S4 behave rather similar. Both compounds are dominated by ferromagnetic exchange, and although HgCr2S4 is an antiferromagnet, no phonon splitting can be observed at the magnetic phase transition. Temperature and magnetic-field dependencies of the eigenfrequencies show no anomalies, indicating displacive polar soft-mode behavior. However, significant effects are detected in the temperature dependence of the plasma frequencies, indicating changes in the nature of the bonds and significant charge transfer. In HgCr2S4, we provide experimental evidence that the magnetic-field dependence of specific polar modes reveals shifts exactly correlated with the magnetization, showing significant magnetodielectric effects even at infrared frequencies.