Fitness and education may protect against cognitive impairments in aging. They may also counteract age-related structural changes within the brain. Here we analyzed volumetric differences in cerebrospinal fluid and gray and white matter, along with neuropsychological data, in adults differing in age, fitness, and education. Cognitive performance was correlated with fitness and education. Voxel-based morphometry was used for a whole-brain analysis of structural magnetic resonance images. We found age-related losses in gray and white matter in medial-temporal, parietal, and frontal areas. As in previous work, fitness within the old correlated with preserved gray matter in the same areas. In contrast, higher education predicted preserved white matter in inferior frontal areas. These data suggest that fitness and education may both be predictive of preserved cognitive function in aging through separable effects on brain structure.