Purpose Gene panel sequencing is revolutionizing germline risk assessment for hereditary breast cancer. Despite scant evidence supporting the role of many of these genes in breast cancer predisposition, results are often reported to families as the definitive explanation for their family history. We assessed the frequency of mutations in 18 genes included in hereditary breast cancer panels among index cases from families with breast cancer and matched population controls. Patients and Methods Cases (n = 2,000) were predominantly breast cancer-affected women referred to specialized Familial Cancer Centers on the basis of a strong family history of breast cancer and BRCA1 and BRCA2 wild type. Controls (n = 1,997) were cancer-free women from the LifePool study. Sequencing data were filtered for known pathogenic or novel loss-of-function mutations. Results Excluding 19 mutations identified in BRCA1 and BRCA2 among the cases and controls, a total of 78 cases (3.9%) and 33 controls (1.6%) were found to carry potentially actionable mutations. A significant excess of mutations was only observed for PALB2 (26 cases, four controls) and TP53 (five cases, zero controls), whereas no mutations were identified in STK11. Among the remaining genes, loss-of-function mutations were rare, with similar frequency between cases and controls. Conclusion The frequency of mutations in most breast cancer panel genes among individuals selected for possible hereditary breast cancer is low and, in many cases, similar or even lower than that observed among cancer-free population controls. Although multigene panels can significantly aid in cancer risk management and expedite clinical translation of new genes, they equally have the potential to provide clinical misinformation and harm at the individual level if the data are not interpreted cautiously.