The apolipoprotein E (APOE) e4 allele is strongly associated with increased risk of cognitive impairments in older adulthood. There is also a possible link to enhanced cognitive performance in younger adults, and the APOE e4 allele may constitute an example of antagonistic pleiotropy. The aim of this work was to investigate the cognitive and neural (functional) effects of the APOE e4 allele during mid-age (45–55 years), where a transition toward cognitive deficit might be expected. APOE e4 carriers (e4+) were compared with non-e4 carriers (e4−) on tasks of sustained and covert attention and prospective memory, and functional magnetic resonance imaging data acquired. Performance by e4+ was equivalent or better than e4− on all 3 tasks, although performance benefits were less pronounced than in youth. Neurally, e4+ showed less task-related recruitment of extrastriate and parietal areas. This became more evident when neural activation data were compared with that of young adults acquired in a parallel study. As expected, mid-age participants showed more diffuse neural activation. Notable was the fact that e4+ showed a relative inability to recruit parietal regions as they aged. This was coupled with a tendency to show greater recruitment of frontal regions, and underactivation of extrastriate visual regions. Thus, mid-age e4+ show a pattern of neural recruitment usually seen later in life, possibly reflecting the source of an accelerated aging profile that describes the e4 genotype.