IMPORTANCE While brain volume changes are used as surrogate markers for Alzheimer disease neuropathology in clinical studies, the extent to which these changes are due to pathologic features of Alzheimer disease in the aging brain is not well established. This study aims to clarify the neuropathologic correlates of longitudinal brain atrophy. OBJECTIVE To examine the association between brain atrophy during life and neuropathology in an elderly population. DESIGN Autopsy study of a cohort of elderly individuals. SETTING Community-based population. PARTICIPANTS Seventy-one healthy elderly individuals were selected from participants of the Oregon Brain Aging Study for having an autopsy, more than 1 magnetic resonance imaging scan, and the last magnetic resonance imaging scan within 36 months of death. MAIN OUTCOMES AND MEASURES The associations between brain volume trajectories (ventricular, total brain, and hippocampal) and time interaction terms for neurofibrillary tangles, neuritic plaques, gross infarcts, microinfarcts, amyloid angiopathy, Lewy bodies, APOE ϵ4 presence, and clinical diagnosis (no cognitive impairment, mild cognitive impairment, or dementia as time-varying covariates) were examined in mixed-effects models, adjusting for duration of follow-up and age at death. RESULTS Ventricular volume trajectory was significantly associated with age, presence of infarcts, neurofibrillary tangle and neuritic plaque scores, APOE ϵ4 allele presence, and dementia diagnosis. Total brain volume trajectory was significantly associated with age and mild cognitive impairment diagnosis. Hippocampal volume trajectory was significantly associated with amyloid angiopathy. CONCLUSIONS AND RELEVANCE Ventricular volume trajectory is more sensitive than total brain and hippocampal volume trajectories as a marker of accruing Alzheimer disease and vascular pathology in elderly individuals. The association between brain volume trajectories and cognitive impairment (mild cognitive impairment and dementia) remained after controlling for the degree of neuropathology and other covariates. This suggests that there may be other factors not measured in this study that could be contributing to brain atrophy in those with cognitive impairment.