Abstract Purpose Up to 10% of all breast cancers (BC) are attributed to inherited pathogenic variants (PV) in BC susceptibility genes; however, most carriers of PVs remain unidentified. Here, we sought to determine the yield of hereditary cancer gene PVs among diverse women attending breast imaging centers, who could benefit from enhanced surveillance and/or risk reduction interventions. Methods This cross-sectional retrospective cohort study included consecutive women, unselected for personal or family cancer history, who were offered genetic testing for hereditary cancer genes at the time of breast imaging at three centers (November 2020–March 2022). Results Among 1943 patients (median age: 66 years), self-reported race/ethnicity was White (34.5%), Hispanic (27.7%), African American (17.9%), Asian (4.5%), Ashkenazi Jewish (0.6%), Other (3.5%), and missing (13.0%). Thirty-nine patients (2%) were identified as carriers of a PV in an autosomal dominant clinically actionable hereditary breast and ovarian cancer (HBOC)-related or Lynch syndrome gene, most frequently, BRCA2 (6/39; 15.4%), PALB2 (8/39; 20.5%), CHEK2 (10/39; 25.6%), and PMS2 (5/39; 12.8%). Of the 34 PVs with known race/ethnicity, 47% were detected among non-White patients. Overall, 354/1,943 (18.2%) of patients met NCCN guidelines for HBOC gene testing and only 15/39 (38.5%) patients with an autosomal dominant clinically actionable PV met guidelines. Conclusion This population health approach extended the reach of genetic cancer risk assessment in a diverse population and highlighted the limits of a guideline-based approach. This may help address inequity in access to risk-appropriate screening and cancer prevention.