Background and Purpose— Cerebral white matter lesions and lacunar infarcts are small vessel disease-related lesions, which are associated with cognitive decline and dementia. We aimed to assess the relationship between risk factors, effect modifiers, and progression of these lesions. Furthermore, we studied the cognitive consequences of lesion progression. Methods— Six hundred sixty-eight people, aged 60 to 90 years, underwent repeated MRI scanning and neuropsychological testing within 3-year follow-up. We rated incident lacunar infarcts and change in periventricular and subcortical white matter lesion severity with a semiquantitative scale. We assessed the relationships between age, sex, baseline lesion load, risk factors, lesion progression, and change in cognitive function by multivariate regression analyses and additional stratified analyses. Results— Baseline lesion load, higher age, high blood pressure, and current smoking were independently associated with progression of white matter lesions. Women had more marked progression of subcortical white matter lesions and incident lacunar infarcts compared with men. Carotid atherosclerosis was associated with incident lacunar infarcts. Higher blood pressure did not contribute to lesion progression in people with already severe lesions at baseline nor in the very old. Lesion progression was associated with a paralleled decline in general cognitive function and in particular with a decreased information processing speed. Conclusions— Higher age, female sex, cigarette smoking, elevated blood pressure, and baseline lesion load were associated with small vessel disease progression. Age and baseline lesion load influenced the risk relations with blood pressure. Progression of small vessel disease was related to a paralleled decline in cognitive function.