BACKGROUND: While the evidence of an association between the apolipoprotein E (APOE) *E4 allele and Alzheimer's disease is very strong, the effect of the *E4 allele on cognitive decline in the general population is more equivocal. A cross-sectional study on the lifespan effects of the *E4 allele 1 failed to find any effect of the *E4 allele on cognitive performance at ages 20-24, 40-44 or 60-64 years. METHODS: In this four year follow-up study, we reexamine the effect of *E4 in the sample of 2,021 individuals, now aged 65-69 years. RESULTS: Performance on the Mini-Mental State Examination (MMSE) was significantly poorer for *E4 homozygotes than heterozygotes or non-carriers. The effects of the *E4 genotype on cognitive decline over four years were found on the MMSE and Symbol-Digit Modalities test but only when controlling for risk factors such as head injury and education. Analyses were repeated with the exclusion of participants diagnosed with a mild cognitive disorder, with little change. CONCLUSION: It is possible that *E4 carriers become vulnerable to greater cognitive decline in the presence of other risk factors at 65-69 years of age.