Abstract Purpose To determine thresholds for amyloid beta pathology and evaluate associations with longitudinal memory performance with the aim to identify a grey zone of early amyloid beta accumulation and investigate its clinical relevance. Methods We included 162 cognitively normal participants with subjective cognitive decline from the SCIENCe cohort (64 ± 8 years, 38% F, MMSE 29 ± 1). Each underwent a dynamic [¹⁸F] florbetapir PET scan, a T1-weighted MRI scan and longitudinal memory assessments (RAVLT delayed recall, n = 655 examinations). PET scans were visually assessed as amyloid positive/negative. Additionally, we calculated the mean binding potential (BP_ND) and standardized uptake value ratio (SUVr_50–70) for an a priori defined composite region of interest. We determined six amyloid positivity thresholds using various data-driven methods (resulting thresholds: BP_ND 0.19/0.23/0.29; SUVr 1.28/1.34/1.43). We used Cohen’s kappa to analyse concordance between thresholds and visual assessment. Next, we used quantiles to divide the sample into two to five subgroups of equal numbers (median, tertiles, quartiles, quintiles), and operationalized a grey zone as the range between the thresholds (0.19–0.29 BP_ND/1.28–1.43 SUVr). We used linear mixed models to determine associations between thresholds and memory slope. Results As determined by visual assessment, 24% of 162 individuals were amyloid positive. Concordance with visual assessment was comparable but slightly higher for BP_ND thresholds (range kappa 0.65–0.70 versus 0.60–0.63). All thresholds predicted memory decline (range beta − 0.29 to − 0.21, all p < 0.05). Analyses in subgroups showed memory slopes gradually became steeper with higher amyloid load (all p for trend < 0.05). Participants with a low amyloid burden benefited from a practice effect (i.e. increase in memory), whilst high amyloid burden was associated with memory decline. Memory slopes of individuals in the grey zone were intermediate. Conclusion We provide evidence that not only high but also grey zone amyloid burden subtly impacts memory function. Therefore, in case a binary classification is required, we suggest using a relatively low threshold which includes grey zone amyloid pathology.