Background: High-dose ionizing radiation (IR) (>0.5 Gy) is an established risk factor for cognitive impairments, but this cannot be concluded for low-to-moderate IR exposure (<0.5 Gy) in adulthood as study results are inconsistent. The objectives are to summarize relevant epidemiological studies of low-to-moderate IR exposure in adulthood and to assess the risk of non-cancerous CNS diseases. Methods: A systematic literature search of four electronic databases was performed to retrieve relevant epidemiological studies published from 2000 to 2022. Pooled standardized mortality ratios, relative risks, and excess relative risks (ERR) were estimated with a random effect model. Results: Forty-five publications were included in the systematic review, including thirty-three in the quantitative meta-analysis. The following sources of IR-exposure were considered: atomic bomb, occupational, environmental, and medical exposure. Increased dose-risk relationships were found for cerebrovascular diseases incidence and mortality (ERRpooled per 100 mGy = 0.04; 95% CI: 0.03–0.05; ERRpooled at 100 mGy = 0.01; 95% CI: −0.00–0.02, respectively) and for Parkinson’s disease (ERRpooled at 100 mGy = 0.11; 95% CI: 0.06–0.16); Conclusions: Our findings suggest that adult low-to-moderate IR exposure may have effects on non-cancerous CNS diseases. Further research addressing inherent variation issues is encouraged.