Bariatric surgery prior to pregnancy is a significant risk factor for small for gestational age (SGA) babies. This case-control study investigated differences between mothers delivering an SGA baby following bariatric surgery, compared to those delivering an appropriate for gestational age (AGA) baby. Out of 129 babies born to mothers in the AURORA cohort study, 25 were SGA (<10th percentile) and 97 were AGA (10th–90th percentile). Higher gestational weight gain (GWG) was significantly associated with decreased odds of SGA (aOR per kg 0.92, 95% CI 0.85–0.99). According to the Institute of Medicine GWG guidelines, 44% of SGA mothers had ‘inadequate’ GWG compared to 17% of AGA mothers. Nearly half of the mothers had ‘excessive’ GWG yet still gave birth to an SGA or AGA baby. Mothers of SGA babies lost more weight following bariatric surgery (45.6 ± 14.4 kg vs. 39.0 ± 17.9 kg). Women who reported receiving nutritional advice following bariatric surgery were significantly less likely to have an SGA baby (aOR 0.15, 95% CI 0.0.4–0.55). Women with a history of bariatric surgery should be provided with specialized support before and during pregnancy to encourage adequate nutritional intake and weight gain to support healthy fetal growth.