Slowed processing speed is common in elderly subjects and frequently related to cerebral small vessel disease. Previous studies have demonstrated associations between processing speed and subcortical ischemic lesions as well as cortical alterations but the precise functional-anatomical relationships remain poorly understood. Here we assessed the impact of both cortical and subcortical changes on processing speed by measuring regional cortical thickness and regional lesion volumes within distinct white-matter tracts. To limit confounding effects from age-related pathologies we studied patients with CADASIL, a genetic small vessel disease. General linear model analysis revealed significant associations between cortical thickness in the medial frontal and occipito-temporal cortex and processing speed. Bayesian network analysis showed a robust conditional dependency between the volume of lacunar lesions in the left anterior thalamic radiation and cortical thickness of the left medial frontal cortex, and between thickness of the left medial frontal cortex and processing speed, whereas there was no direct dependency between lesion volumes in the left anterior thalamic radiation and processing speed. Our results suggest that the medial frontal cortex has an intermediate position between lacunar lesions in the anterior thalamic radiation and deficits in processing speed. In contrast, we did not observe such a relationship for the occipito-temporal region. These findings reinforce the key role of frontal-subcortical circuits in cognitive impairment resulting from cerebral small vessel disease.