Cognitive impairment in older Latinos is of concern due to the rapid growth of this population and their increased risk for dementia due to chronic disease. Evidence, primarily from studies of non-Latino Whites, suggests that physical activity (PA) may reduce cognitive decline. Few longitudinal studies have included older Latinos, objective measures of PA, or neurocognitive tests that assess domains of cognition. The purpose of this longitudinal study was to explore the relationship between changes in PA and cognitive decline in older Latinos over an average of 5 years. Inclusion criteria for the baseline sample were age ≥50 years, Latino ethnicity (English or Spanish speaking), no ambulation disability, no evidence of dementia, and Chicago address. Of the 174 baseline participants, 59 (33.9%) participated at follow-up. PA was measured by questionnaire and accelerometer worn for 7 days. A battery of neurocognitive tests assessed episodic memory, perceptual speed, and semantic memory. Change in cognitive function was dichotomized to maintenance versus decline. Binary logistic regression results indicated that those who had less decline from baseline to follow-up in self-reported light PA maintained episodic memory, odds ratio ( OR) = 1.16 (95% confidence interval [CI] [1.03, 1.32]), while those who had less decline in accelerometer moderate–vigorous bouts maintained semantic memory, OR = 16.08 (95% CI [1.53, 168.89]), controlling for baseline age, chronic health problems, depressive symptoms, and acculturation. These findings suggest that maintenance of PA with aging may prevent cognitive decline. This work can inform future intervention development that aims to maintain PA in order to prevent cognitive decline.