Infections during pregnancy can seriously damage fetal neurodevelopment by aberrantly activating the maternal immune system, directly impacting fetal neural cells. Increasing evidence suggests that these adverse impacts involve alterations in neural stem cell biology with long-term consequences for offspring, including neurodevelopmental disorders such as autism spectrum disorder, schizophrenia, and cognitive impairment. Here we review how maternal infection with viruses such as Influenza A, Cytomegalovirus, and Zika during pregnancy can affect the brain development of offspring by promoting the release of maternal pro-inflammatory cytokines, triggering neuroinflammation of the fetal brain, and/or directly infecting fetal neural cells. In addition, we review insights into how these infections impact human brain development from studies with animal models and brain organoids. Finally, we discuss how maternal infection with SARS-CoV-2 may have consequences for neurodevelopment of the offspring.