Sagnac interferometry can provide a substantial improvement in signal-to-noise ratio compared to conventional magnetic imaging based on the magneto-optical Kerr effect. We show that this improvement is sufficient to allow quantitative measurements of current-induced magnetic deflections due to spin-orbit torque even in thin-film magnetic samples with perpendicular magnetic anisotropy, for which the Kerr rotation is second order in the magnetic deflection. Sagnac interferometry can also be applied beneficially for samples with in-plane anisotropy, for which the Kerr rotation is first order in the deflection angle. Optical measurements based on Sagnac interferometry can therefore provide a cross-check on electrical techniques for measuring spin-orbit torque. Different electrical techniques commonly give quantitatively inconsistent results so that Sagnac interferometry can help to identify which techniques are affected by unidentified artifacts.