The World Health Organization (WHO) has warned that a large majority of children do not reach its recommendations on physical activity for health, i.e., 60 min a day of moderate to vigorous physical activity. The objective of this study was to know the index of habitual physical activity in different contexts where the child interacts: school index (SCHOOL-I), extracurricular/sports activity index (SPORT-I), free time index (LEISURE-I) and the total habitual physical activity index (GLOBAL-I) based on age and gender. 900 Primary Education schoolchildren from Galicia (Spain) aged 10–12 years (M = 10.84; SD = 0.67) participated, of which 454 (50.40%) were boys and 446 (49.6%) girls. For data collection, the validated Inventory of Habitual Physical Activity in Schoolchildren (IAFHE) questionnaire was used. The results show a significant main effect on the age factor in SCHOOL-I (p < 0.001), and in GLOBAL-I (p = 0.034), the rates being higher in 10-year-old children compared to 11 and 12. A significant main effect has also been found in the gender factor in SCHOOL-I (p < 0.001), SPORT-I (p < 0.001) and in GLOBAL-I (p < 0.001), being greater in boys than in girls. It is concluded that, as school age increases, a lower index of physical activity is registered, this being higher in boys compared to girls. It is necessary to establish strategies to promote the practice of physical activity from different agents and in different contexts.