Handgrip strength is a widely used measure of muscle strength and a predictor of a range of morbidities including cardiovascular diseases and all-cause mortality. Previous genome-wide association studies of handgrip strength have focused on common variants primarily in persons of European descent. We aimed to identify rare and ancestry-specific genetic variants associated with handgrip strength by conducting whole-genome sequence association analyses using 13,552 participants from six studies representing diverse population groups from the Trans-Omics in Precision Medicine (TOPMed) Program. By leveraging multiple handgrip strength measures performed in study participants over time, we increased our effective sample size by 7–12%. Single-variant analyses identified ten handgrip strength loci among African-Americans: four rare variants, five low-frequency variants, and one common variant. One significant and four suggestive genes were identified associated with handgrip strength when aggregating rare and functional variants; all associations were ancestry-specific. We additionally leveraged the different ancestries available in the UK Biobank to further explore the ancestry-specific association signals from the single-variant association analyses. In conclusion, our study identified 11 new loci associated with handgrip strength with rare and/or ancestry-specific genetic variations, highlighting the added value of whole-genome sequencing in diverse samples. Several of the associations identified using single-variant or aggregate analyses lie in genes with a function relevant to the brain or muscle or were reported to be associated with muscle or age-related traits. Further studies in samples with sequence data and diverse ancestries are needed to confirm these findings.