Various signs on routine brain MRI can help differentiate between Parkinson’s disease (PD) and the various forms of atypical parkinsonism (AP). Here, we evaluate what routine brain MRI contributes to the clinical diagnosis, in both early and advanced disease stages. We performed a prospective observational study in 113 patients with parkinsonism, but without definite diagnosis upon inclusion. At baseline, patients received a structured interview, comprehensive and standardized neurological assessment, and brain MRI. The silver standard diagnosis was made after 3 years of follow-up (PD n = 43, AP n = 57), which was based on disease progression, repeat standardized neurological examination and response to treatment. The clinical diagnosis was classified as having either ‘low certainty’ (lower than 80%) or ‘high certainty’ (80% or higher). The added diagnostic yield of baseline MRI results were then studied relative to clinical neurological evaluation at presentation, and at follow-up. Sensitivity and specificity for separating AP from PD were calculated for all potentially distinguishing MRI abnormalities described previously in the literature. MRI abnormalities showed moderate to high specificity but limited sensitivity for the diagnosis of AP. These MRI abnormalities contributed little over and above the clinically based diagnosis, except when the clinical diagnosis was uncertain. For these patients, presence of putaminal or cerebellar atrophy was particularly indicative of AP. Routine brain MRI has limited added value for differentiating between PD and AP when clinical certainty is already high, but has some diagnostic value when the clinical diagnosis is still uncertain.