Background Magnetic resonance imaging‐visible perivascular spaces ( PVS ) are related to interstitial fluid clearance pathways (including amyloid‐β) in the brain and are suggested to be a marker of cerebral small vessel disease. We investigated the role, topography, and possible implications of PVS in cognitive impairment. Methods and Results A total of 1504 patients undergoing memory clinic investigation and an associated brain magnetic resonance imaging scan were included in this cross‐sectional study. Magnetic resonance images were assessed for markers of small vessel disease. Additionally, 1039 patients had cerebrospinal fluid analysis of amyloid‐β 42, total tau (T‐tau), and phosphorylated tau ( P ‐tau); 520 patients had apoE genotyping done. Results were analyzed with generalized linear models. A total of 289 (19%; 95% confidence interval, 17–21) had a high‐grade PVS in the centrum semiovale ( CSO ) and 65 (4%; 95% confidence interval: 3%–5%) in the basal ganglia ( BG ). Centrum semiovale– and BG ‐ PVS were both associated with high age ( P <0.001), hypertension ( P <0.001), probable cerebral amyloid angiopathy ( P <0.05), moderate‐to‐severe white matter hyperintensities ( P <0.001), cortical superficial siderosis ( P <0.001), cerebral microbleeds ( P <0.001), and PVS . centrum semiovale– PVS was separately associated with strictly lobar cerebral microbleeds ( P =0.057). BG ‐ PVS was associated with strictly deep cerebral microbleeds ( P <0.001), lacunes ( P <0.001), and vascular dementia ( P =0.04). BG ‐ PVS showed a tendency to be associated with high cerebrospinal fluid tau (B=0.002, P =0.04) in the whole cohort and in Alzheimer's disease (B=0.005; P =0.02). No other associations with cerebrospinal fluid or the apoE e4 allele was observed. Conclusions Centrum semiovale– PVS and BG ‐ PVS have different underlying etiology, being associated with cerebral amyloid angiopathy and hypertensive vasculopathy, respectively, although a significant overlap between these pathologies is likely to exist.