The human skeletal form underlies bipedalism, but the genetic basis of skeletal proportions (SPs) is not well characterized. We applied deep-learning models to 31,221 x-rays from the UK Biobank to extract a comprehensive set of SPs, which were associated with 145 independent loci genome-wide. Structural equation modeling suggested that limb proportions exhibited strong genetic sharing but were independent of width and torso proportions. Polygenic score analysis identified specific associations between osteoarthritis and hip and knee SPs. In contrast to other traits, SP loci were enriched in human accelerated regions and in regulatory elements of genes that are differentially expressed between humans and great apes. Combined, our work identifies specific genetic variants that affect the skeletal form and ties a major evolutionary facet of human anatomical change to pathogenesis.