The objectives were to clarify whether the relationship between physical performance and frailty was independently and jointly mediated by movement behaviors and body composition. We analyzed 871 older adults (476 women) from The Toledo Study for Healthy Aging. Skeletal muscle index (SMI) and fat index (FI) were determined using bone densitometry. Sedentary time (ST) and moderate-to-vigorous physical activity (MVPA) were assessed using accelerometry. The Frailty Trait Scale and The Short Physical Performance Battery (SPPB) were used to evaluate frailty and physical performance, respectively. Simple and multiple mediation analyses were carried out to determine the role of movement behaviors and body composition, adjusted for potential confounders. ST and MVPA acted independently as mediators in the relationship between SPPB and frailty (0.06% for ST and 16.89% for MVPA). FI also acted as an independent mediator in the same relationship (36.47%), while the mediation role of SMI was not significant. MVPA and FI both acted jointly as mediators in this previous relationship explaining 58.15% of the model. Our data support the fact that interventions should simultaneously encourage the promotion of MVPA and strategies to decrease the FI in order to prevent or treat frailty through physical performance improvement.