Self-rated health (SRH) is an independent determinant for all-cause mortality. We aimed to examine the independent and combined associations of components of physical fitness with SRH at baseline (cross-sectional) and two years later (longitudinal) in children and adolescents. Spanish youth (N = 1378) aged 8 to 17.9 years participated at baseline. The dropout rate at 2-year follow-up was 19.5% (n = 270). Participants were categorized as either children (8 to 11.9 years age) or adolescents (12 to 17.9 years age). The ALPHA health- related fitness test battery for youth was used to assess physical fitness, and SRH was measured by a single-item question. Cumulative link, ANOVA and ANCOVA models were fitted to analyze the data. Cardiorespiratory fitness, relative upper body isometric muscular strength, muscular strength score, and global physical fitness were positively associated with SRH in children (OR, 1.048; 95% CI, 1.020–1.076; OR, 18.921; 95% CI, 3.47–104.355; OR, 1.213; 95% CI, 1.117–1.319, and OR, 1.170; 95% CI, 1.081–1.266, respectively; all p < 0.001) and adolescents (OR, 1.057; 95% CI, 1.037–1.076; OR, 5.707; 95% CI, 1.122–29.205; OR, 1.169; 95% CI, 1.070–1.278, and OR, 1.154 95% CI, 1.100–1.210, respectively; all p < 0.001); and motor fitness was positively associated with SRH only in adolescents at baseline (OR, 1.192; 95% CI, 1.066–1.309; p < 0.01). Cardiorespiratory fitness and global physical fitness were positively associated with SRH in children two years later (OR, 1.056; 95% CI, 1.023–1.091; p < 0.001; and OR, 1.082; 95% CI, 1.031–1.136; p < 0.01; respectively). Only cardiorespiratory fitness was independently associated with SRH in children and adolescents at baseline (OR, 1.059; 95% CI, 1.029–1.090; and OR, 1.073; 95% CI, 1.050–1.097, respectively; both p < 0.001) and two years later (OR, 1.075; 95% CI, 1.040–1.112; p < 0.001; and OR, 1.043; 95% CI, 1.014–1.074; p < 0.01, respectively). A high level of cardiorespiratory fitness at baseline or maintaining high levels of cardiorespiratory fitness from the baseline to 2-year follow-up were associated with a higher level of SRH at 2-year follow-up in children (p < 0.01) and adolescents (p < 0.05). These findings emphasize the importance of cardiorespiratory fitness as strong predictor of present and future SRH in youth. Intervention programs to enhance cardiorespiratory fitness level of the youth population are urgently needed for present and future youth’s health.