Abstract Background Early detection of individuals at risk for Alzheimer’s disease (AD) is highly important. Amyloid accumulation is an early pathological AD event, but the genetic association with known AD risk variants beyond the APOE4 effect is largely unknown. We investigated the association between different AD polygenic risk scores (PRS) and amyloid accumulation in the Flemish Prevent AD Cohort KU Leuven (F-PACK). Methods We calculated PRS with and without the APOE region in 90 cognitively healthy F-PACK participants (baseline age 67.8 (52–80) years, 41 APOE4 carriers), with baseline and follow-up amyloid-PET (time interval 6.1 (3.4–10.9) years). Individuals were genotyped using Illumina GSA and imputed. PRS were calculated using three p-value thresholds (pT) for variant inclusion: 5 × 10⁻⁸, 1 × 10⁻⁵, and 0.1, based on the stage 1 summary statistics from Kunkle et al. (Nat Genet 51:414–30, 2019). Linear regression models determined if these PRS predicted amyloid accumulation. Results A score based on PRS excluding the APOE region at pT = 5 × 10⁻⁸ plus the weighted sum of the two major APOE variants (rs429358 and rs7412) was significantly associated with amyloid accumulation (p = 0.0126). The two major APOE variants were also significantly associated with amyloid accumulation (p = 0.0496). The other PRS were not significant. Conclusions Specific PRS are associated with amyloid accumulation in the asymptomatic phase of AD.