Abstract Aims The association between atrial fibrillation (AF) and dementia is well documented, but it is not clear if oral anticoagulant treatment offers protection. The aim of the study is therefore to compare the incidence of new dementia in patients with AF with and without oral anticoagulants, and to explore if there is a difference between novel anticoagulants and warfarin in this respect. Methods and results Retrospective registry study of all patients with hospital diagnosis of AF and no previous diagnosis of dementia in Sweden between 2006 and 2014. Propensity score matching, falsification endpoints, and analyses according to intention to treat as well as on-treatment principles were used. The study included 444 106 patients and over 1.5 million years at risk. Patients on anticoagulant treatment at baseline was associated with 29% lower risk of dementia than patients without anticoagulant treatment [hazard ratio (HR) 0.71, 95% confidence intervals (95% CI) 0.68–0.74] and 48% lower risk analysed on treatment (HR 0.52, 95% CI 0.50–055). Direct comparison between new oral anticoagulants and warfarin showed no difference (HR 0.97, 95% CI 0.67–1.40). Conclusion The risk of dementia is higher without oral anticoagulant treatment in patients with AF. This suggests that early initiation of anticoagulant treatment in patients with AF could be of value in order to preserve cognitive function.