In 2003, we remeasured the frequency of the 1 S ( F = 1, m F = ± 1) 2 S ( F ' = 1, m F ' = ± 1) two- photon transition in atomic hydrogen with a self-referenced optical frequency comb relative to the 133 Cs ground-state hyperfine splitting. By comparison with an earlier measurement made in 1999, we can set an upper limit on its time variation of -29(57) Hz within 44 months. Combining this result with recently published abso- lute frequency measurements of optical transitions in single trapped Hg + and Yb + ions, we deduce ( ∂ / ∂ t )ln α = (-0.3 ± 2.0) × 10 -15 yr -1 and ( ∂ / ∂ t )ln( µ Cs / µ B ) = (2.4 ± 6.8) × 10 -15 yr -1 as limits on the fractional time variation of the fine structure constant α and the ratio of the 133 Cs nuclear magnetic moment µ Cs and the Bohr magneton µ B , respectively. Using frequency comparisons of microwave 87 Rb and 133 Cs fountain clocks, we deduce addi- tional stringent limits on the fractional time variation of µ Rb / µ Cs and µ Rb / µ B , where µ Rb is the magnetic moment of the 87 Rb nucleus. All limits are compatible with zero drift and, for the first time, do not require any model assumptions about the relative drifts of fundamental constants.