Abstract Purpose To examine associations between the APOE-ε2 and APOE-ε4 alleles and core Alzheimer’s disease (AD) pathological hallmarks as measured by amyloid-β (Aβ) and tau PET in older individuals without dementia. Methods We analyzed data from 462 ADNI participants without dementia who underwent Aβ ([¹⁸F]florbetapir or [¹⁸F]florbetaben) and tau ([¹⁸F]flortaucipir) PET, structural MRI, and cognitive testing. Employing APOE-ε3 homozygotes as the reference group, associations between APOE-ε2 and APOE-ε4 carriership with global Aβ PET and regional tau PET measures (entorhinal cortex (ERC), inferior temporal cortex, and Braak-V/VI neocortical composite regions) were investigated using linear regression models. In a subset of 156 participants, we also investigated associations between APOE genotype and regional tau accumulation over time using linear mixed models. Finally, we assessed whether Aβ mediated the cross-sectional and longitudinal associations between APOE genotype and tau. Results Compared to APOE-ε3 homozygotes, APOE-ε2 carriers had lower global Aβ burden (β_std [95% confidence interval (CI)]: − 0.31 [− 0.45, − 0.16], p = 0.034) but did not differ on regional tau burden or tau accumulation over time. APOE-ε4 participants showed higher Aβ (β_std [95%CI]: 0.64 [0.42, 0.82], p < 0.001) and tau burden (β_std range: 0.27-0.51, all p < 0.006). In mediation analyses, APOE-ε4 only retained an Aβ-independent effect on tau in the ERC. APOE-ε4 showed a trend towards increased tau accumulation over time in Braak-V/VI compared to APOE-ε3 homozygotes (β_std [95%CI]: 0.10 [− 0.02, 0.18], p = 0.11), and this association was fully mediated by baseline Aβ. Conclusion Our data suggest that the established protective effect of the APOE-ε2 allele against developing clinical AD is primarily linked to resistance against Aβ deposition rather than tau pathology.