A positive perception of motor skills is important for physical activity participation. The aim was to investigate which modifiable factors predict children’s perceived motor skills. Mothers completed questionnaires when their child was 3.5 and 5 years old. At 5 years old, the children’s perceived motor competence (PMC) was assessed. Separate linear regression models (up to 300 children) examined which factors at each time point predicted children’s PMC, adjusted for relevant confounders. Multivariate models were then run with factors associated (p < 0.10) with perception. At 3.5 years, the time spent with same age and older children (both higher tertiles) and parental physical activity facilitation (sum of facilitation in last month, e.g., taking child to park) were initially associated with higher perception. Dance/gymnastics participation were associated with lower perceptions. Other child behaviours, maternal beliefs, play equipment, and swimming lessons were non-significant. In the final prospective model (n = 226), parental physical activity facilitation when child was 3.5 years old was the only factor to predict PMC. No factors were significant for the cross-sectional analyses at 5 years. Perceptions are formed based on past experiences which may explain why factors at 3.5 years rather than current experiences (when children were 5 years old) were associated with childhood perceptions.