Background: Cognitive decline is characterized in multiple sclerosis (MS), but the rate and severity vary. The reserve hypothesis proposes that baseline neurological differences impact cognitive outcome in neurodegenerative disease. Objective: To elucidate how brain reserve and cognitive reserve influence subcortical gray matter (SCGM) atrophy and cognitive decline in MS over 3 years. Methods: Seventy-one MS patients and 23 normal controls underwent magnetic resonance imaging and cognitive assessment at baseline and 3-year follow-up. The influence of reserve on cognitive processing speed (CPS) and memory was examined. Results: SCGM volume and cognitive scores were lower in MS than normal controls ( P⩽0.001). Accounting for baseline, comparison of follow-up means yielded a difference between groups in SCGM volume ( P<0.001) but not cognition (NS). Cognitive reserve ( P=0.005), but not brain reserve, contributed to CPS, with only low cognitive reserve MS subjects showing decline in CPS ( P=0.029). SCGM change predicted CPS outcome in MS with low cognitive reserve ( P=0.002) but not high cognitive reserve. There were no effects in the domain of memory. Conclusions: SCGM atrophy occurs in normal controls, but significantly more so in MS. While CPS did not change in normal controls, low cognitive reserve was associated with CPS decline in MS. High cognitive reserve protect MS patients from cognitive decline related to SCGM atrophy.