This study assessed relationships among white matter hyperintensities (WMH), cerebrospinal fluid (CSF), Alzheimer's disease (AD) pathology markers, and brain volume loss. Subjects included 197 controls, 331 individuals with mild cognitive impairment (MCI), and 146 individuals with AD with serial volumetric 1.5-T MRI. CSF Aβ1-42 (n = 351) and tau (n = 346) were measured. Brain volume change was quantified using the boundary shift integral (BSI). We assessed the association between baseline WMH volume and annualized BSI, adjusting for intracranial volume. We also performed multiple regression analyses in the CSF subset, assessing the relationships of WMH and Aβ1-42 and/or tau with BSI. WMH burden was positively associated with BSI in controls (p = 0.02) but not MCI or AD. In multivariable models, WMH (p = 0.003) and Aβ1-42 (p = 0.001) were independently associated with BSI in controls; in MCI Aβ1-42 (p < 0.001) and tau (p = 0.04) were associated with BSI. There was no evidence of independent effects of WMH or CSF measures on BSI in AD. These data support findings that vascular damage is associated with increased brain atrophy in the context of AD pathology in pre-dementia stages.