Physical activity (PA) reduces the risk of cognitive decline (CD) in the general population. However, little is known about whether the presence of the apolipoprotein E epsilon 4 allele (APOE e4) could modify this beneficial effect. The aim of this systematic review was to analyze and synthetize the scientific evidence related to PA levels and CD risk in cognitively healthy APOE e4 carriers. Four electronic databases were analyzed. Only original articles with longitudinal study design were selected to analyze the relationship between PA and CD in APOE e4 carriers. Five studies were included in the systematic review. All studies except one stated that PA is a protective factor against CD in APOE e4 carriers. Moreover, partial support was found for the hypothesis that a greater amount and intensity of PA are more beneficial in CD prevention. The results support the idea that PA is a protective factor against CD in APOE e4 carriers. Nevertheless, it would be necessary to carry out further studies that would allow these findings to be contrasted.