Abstract Background Physical activity has been suggested to prevent the conversion of mild cognitive impairment (MCI) to dementia in patients. We investigated the association between the continuance and regularity of physical activity and the risk of developing dementia in patients with MCI. Methods We analyzed 6-year followed up data for 247,149 individuals in the National Health Insurance Service (NHIS) cohort of Korea who were enrolled between January 1, 2009, and December 31, 2015. The patients were divided into four groups: those who did not engage in physical activity consistently (Never-PA group), those who initiated physical activity (Initiation-PA group), those who ceased physical activity (Withdrawal-PA group), and those who performed physical activity consistently (Maintenance-PA group). We also divided the patients into two groups: those who engaged in physical activity irregularly (Irregular-PA) and those who undertook physical activity regularly (Regular-PA). Results When the risk for the Never-PA group was set as the benchmark (ref = 1), the Maintenance-PA group had the lowest incidence of dementia of the Alzheimer type (DAT) compared to the other groups (HR = 0.82, 95% CI 0.79–0.86). The DAT risk of the Initiation-PA group (HR = 0.89, 95% CI 0.85–0.93) was lower than the Never-PA group. In addition, compared to the Irregular-PA group, the Regular-PA group had a 15% reduced risk for developing DAT. Conclusions Although no causal inference could be made, continued regular physical activity in patients with MCI is associated with a protective effect against developing DAT. Moreover, ceasing physical activity could halt this protective effect.