Cohort studies that quantify volumetric brain data among individuals with different levels of COVID-19 severity are presently limited. It is still uncertain whether there exists a potential correlation between disease severity and the effects of COVID-19 on brain integrity. Our objective was to assess the potential impact of COVID-19 on measured brain volume in patients with asymptomatic/mild and severe disease after recovery from infection, compared with healthy controls, using artificial intelligence (AI)-based MRI volumetry. A total of 155 participants were prospectively enrolled in this IRB-approved analysis of three cohorts with a mild course of COVID-19 (n = 51, MILD), a severe hospitalised course (n = 48, SEV), and healthy controls (n = 56, CTL) all undergoing a standardised MRI protocol of the brain. Automated AI-based determination of various brain volumes in mL and calculation of normalised percentiles of brain volume was performed with mdbrain software, using a 3D T1-weighted magnetisation-prepared rapid gradient echo (MPRAGE) sequence. The automatically measured brain volumes and percentiles were analysed for differences between groups. The estimated influence of COVID-19 and demographic/clinical variables on brain volume was determined using multivariate analysis. There were statistically significant differences in measured brain volumes and percentiles of various brain regions among groups, even after the exclusion of patients undergoing intensive care, with significant volume reductions in COVID-19 patients, which increased with disease severity (SEV > MILD > CTL) and mainly affected the supratentorial grey matter, frontal and parietal lobes, and right thalamus. Severe COVID-19 infection, in addition to established demographic parameters such as age and sex, was a significant predictor of brain volume loss upon multivariate analysis. In conclusion, neocortical brain degeneration was detected in patients who had recovered from SARS-CoV-2 infection compared to healthy controls, worsening with greater initial COVID-19 severity and mainly affecting the fronto-parietal brain and right thalamus, regardless of ICU treatment. This suggests a direct link between COVID-19 infection and subsequent brain atrophy, which may have major implications for clinical management and future cognitive rehabilitation strategies.