Mercury is highly toxic metal found in trace quantities in common foods. There is concern that exposure during pregnancy could impair infant development. Epidemiological evidence is mixed, but few studies have examined postnatal growth. Differences in nutrition, exposures, and the living environment after birth may make it easier to detect a negative impact from mercury toxicity on infant growth. This study includes 544 mother–child pairs from the Avon Longitudinal Study of Parents and Children. Blood mercury was measured in early pregnancy and infant weight at 10 intervals between 4 and 61 months. Mixed-effect models were used to estimate the change in infant weight associated with prenatal mercury exposure. The estimated difference in monthly weight gain was −0.02 kg per 1 standard deviation increase in Hg (95% confidence intervals: −0.10 to 0.06 kg). When restricted to the 10th decile of Hg, the association with weight at each age level was consistently negative but with wide confidence intervals. The lack of evidence for an association may indicate that at Hg levels in this cohort (median 1.9 µg/L) there is minimal biological impact, and the effect is too small to be either clinically relevant or detectable.