Age-related dysfunctions in cholinergic and dopaminergic neuromodulation are assumed to contribute to age-associated impairment of explicit memory. Both neurotransmitters also modulate attention, working memory, and processing speed. To date, in vivo evidence linking structural age-related changes in these neuromodulatory systems to dysfunction within or across these cognitive domains remains scarce. Using a factor analytical approach in a cross-sectional study including 86 healthy older (aged 55 to 83 years) and 24 young (aged 18 to 30 years) adults, we assessed the relationship between structural integrity-as measured by magnetization transfer ratio (MTR)-of the substantia nigra/ventral tegmental area (SN/VTA), main origin of dopaminergic projections, basal forebrain (major origin of cortical cholinergic projections), frontal white matter (FWM), and hippocampus to neuropsychological and psychosocial scores. Basal forebrain MTR and FWM changes correlated with a factor combining verbal learning and memory and working memory and, as indicated by measures of diffusion, were most likely due to vascular pathology. These findings suggest that frontal white matter integrity and cholinergic neuromodulation provide clues as to why age-related cognitive decline is often correlated across cognitive domains.