In the general population, it is unknown whether stress-related perinatal factors influence lymphocyte subset counts in neonates. The aim of this study was to assess the associations of perinatal factors related to stress and hypoxia (mode of delivery, Apgar scores, and umbilical cord blood pH) with absolute lymphocyte subset counts (T, B, NK, helper T, cytotoxic T, naïve, memory T) in cord blood of 571 neonates. This study was embedded in a population-based prospective cohort study from fetal life onwards. All models were adjusted for gestational age, birth weight, gender, maternal fever, and each of the other perinatal stress-relating factors. Our results showed that increasing stress-related mode of delivery was positively associated with NK and memory T-lymphocyte subset counts (all p < 0.01). Effects of Apgar scores on lymphocyte subsets were explained by umbilical cord blood pH. Lower umbilical cord blood pH was associated with higher B, NK, and memory T-lymphocyte counts (all p < 0.05). Effects of mode of delivery and umbilical cord blood pH on other lymphocyte subsets were not observed. We conclude that, in the general population, lymphocyte subset counts in neonates increase with increasing stress- and hypoxia-related perinatal factors.