Background: Vascular risk factors (VRF) in multiple sclerosis (MS) patients have been associated with lower brain volumes. It is currently unknown if this association already exists in early MS and how it develops over time. Methods: We identified 82 patients with clinically isolated syndrome (CIS) ( n = 61) or with early relapsing-remitting MS ( n = 21) and assessed their VRF including arterial hypertension, hyperlipidaemia, diabetes mellitus and smoking. We analysed T2-lesion load, normalized brain volume (NBV), cortical grey (cGMV) and white matter volumes (WMV), thalamic and basal ganglia volumes at baseline and follow-up magnetic resonance imaging (MRI) and assessed the percentage of brain volume change (PBVC) using SIENA. Results: Patient mean age was 32.4 (±8.7) years and 54 (65%) were women. Median follow-up period was 42 (29–54) months. In total, 26 patients (31.7%) had one or more VRF (VRF+). At baseline, VRF+ patients had a lower NBV (1530.9 cm³ vs 1591.2 cm³, p = 0.001), a lower cGMV (628.5 cm³ vs 668.6 cm³, p = 0.002) and WMV (752.2 cm³ vs 783.9 cm³, p = 0.009) than VRF-negative patients. Similar results were obtained at follow-up. PBVC was comparable between patients with and without VRF. Conclusion: VRF are associated with lower brain volume already in early MS but do not lead to increased brain volume loss during 3.5 years of follow-up.