The aim of this study was to estimate the heritability of and environmental contributions to skeletal muscle phenotypes (appendicular lean mass and calf muscle cross-sectional area) in subjects of African descent and to determine whether heritability estimates are impacted by sex or age. Body composition was measured by dual-energy X-ray absorptiometry and computed tomography in 444 men and women aged 18 yr and older (mean: 43 yr) from eight large, multigenerational Afro-Caribbean families (family size range: 21–112). Using quantitative genetic methods, we estimated heritability and the association of anthropometric, lifestyle, and medical variables with skeletal muscle phenotypes. In the overall group, we estimated the heritability of lean mass and calf muscle cross-sectional area (h² = 0.18–0.23, P < 0.01) and contribution of environmental factors to these phenotypes ( r² = 0.27–0.55, P < 0.05). In our age-specific analysis, the heritability of leg lean mass was lower in older vs. younger individuals (h² = 0.05 vs. 0.23, respectively, P = 0.1). Sex was a significant covariate in our models ( P < 0.001), although sex-specific differences in heritability varied depending on the lean mass phenotype analyzed. High genetic correlations (ρ_G = 0.69–0.81; P < 0.01) between different lean mass measures suggest these traits share a large proportion of genetic components. Our results demonstrate the heritability of skeletal muscle traits in individuals of African heritage and that heritability may differ as a function of sex and age. As the loss of skeletal muscle mass is related to metabolic abnormalities, disability, and mortality in older individuals, further research is warranted to identify specific genetic loci that contribute to these traits in general and in a sex- and age-specific manner.