Background: Dementia has been identified as a major predictor of mortality associated with COVID-19. Objective: The objective of this study was to investigate the association between dementia and mortality in COVID-19 inpatients in Germany across a longer interval during the pandemic. Methods: This retrospective study was based on anonymized data from 50 hospitals in Germany and included patients with a confirmed COVID-19 diagnosis hospitalized between March 11, 2020 and July, 20, 2022. The main outcome of the study was the association of mortality during inpatient stays with dementia diagnosis, which was studied using multivariable logistic regression adjusted for age, sex, and comorbidities as well as univariate logistic regression for matched pairs. Results: Of 28,311 patients diagnosed with COVID-19, 11.3% had a diagnosis of dementia. Prior to matching, 26.5% of dementia patients and 11.5% of non-dementia patients died; the difference decreased to 26.5% of dementia versus 21.7% of non-dementia patients within the matched pairs (n = 3,317). This corresponded to an increase in the risk of death associated with dementia (OR = 1.33; 95% CI: 1.16–1.46) in the univariate regression conducted for matched pairs. Conclusion: Although dementia was associated with COVID-19 mortality, the association was weaker than in previously published studies. Further studies are needed to better understand whether and how pre-existing neuropsychiatric conditions such as dementia may impact the course and outcome of COVID-19.