Cerebral small-vessel disease is common in older people and may contribute to the development of dementia. The objective of the present study was to evaluate the relationship between measures of cerebral small-vessel disease on MRI and the rate of decline in specific cognitive domains in participants from the prospective, population-based Rotterdam Scan Study. Participants were 60-90 years of age and free from dementia at baseline in 1995-1996. White matter lesions (WML), cerebral infarcts and generalized brain atrophy were assessed on the baseline MRI. We performed neuropsychological testing at baseline and repeatedly in 1999-2000 and in 2001-2003. We used random-effects models for repeated measures to examine the association between quantitative MRI measures and rate of decline in measures of global cognitive function, information processing speed, executive function and memory. There were a total of 2266 assessments for the 832 participants in the study, with an average time from the initial to last assessment of 5.2 years. Increasing severity of periventricular WML and generalized brain atrophy and the presence of brain infarcts on MRI were associated with a steeper decline in cognitive function. These structural brain changes were specifically associated with decline in information processing speed and executive function. The associations between MRI measures of cerebral small-vessel disease and cognitive decline did not change after additional adjustment for vascular risk factors or depressed mood. After exclusion of participants with an incident stroke, some of the associations of periventricular WML, brain infarcts and generalized brain atrophy with measures of information processing speed and executive function were no longer significant. This may indicate that stroke plays an intermediate role in the relationship between cerebral small-vessel disease and cognitive decline. Our results suggest that in older people cerebral small-vessel disease may contribute to cognitive decline by affecting information processing speed and executive function.