We aimed to examine the association of APOE ε genotype with MRI markers of cerebrovascular disease (CVD): white matter hyperintensities, brain infarcts, and cerebral microbleeds. We performed a systematic review and meta-analysis of 42 cross-sectional or longitudinal studies identified in PubMed from 1966 to June 2012 (n = 29,965). This included unpublished data from 3 population-based studies: 3C-Dijon, Framingham Heart Study, and Sydney Memory and Ageing Study. When necessary, authors were contacted to provide effect estimates for the meta-analysis. APOE ε4 carrier status and APOE ε44 genotype were associated with increasing white matter hyperintensity burden (sample size-weighted z score meta-analysis [meta]-p = 0.0034 and 0.0030) and presence of cerebral microbleeds (meta odds ratio [OR] = 1.24, 95% confidence interval [CI] [1.07, 1.43], p = 0.004, and 1.87 [1.26, 2.78], p = 0.002), especially lobar. APOE ε2 carrier status was associated with increasing white matter hyperintensity load (z score meta-p = 0.00053) and risk of brain infarct (meta OR = 1.41[1.09, 1.81], p = 0.008). APOE ε4 and APOE ε2 were associated with increasing burden in MRI markers for both hemorrhagic and ischemic CVD. While the association of APOE ε4 with an increased burden of CVD could be partly contributing to the relationship between APOE ε4 and AD, APOE ε2 was associated with MRI markers of CVD in the opposite direction compared to AD.