Abstract Background We aimed to evaluate the influence of mammographic breast density at diagnosis on the risk of cancer recurrence and survival outcomes in patients with invasive breast cancer after modified radical mastectomy. Methods This case-control study included 121 case-control pairs of women diagnosed with invasive breast cancer between 2004 and 2009, and who had undergone modified radical mastectomy and had mammographic breast density measured before or at diagnosis. Women with known locoregional recurrence or distant metastasis were matched by pathological disease stage, age, and year of diagnosis to women without recurrence. Locoregional recurrence was defined as recurrence in the ipsilateral chest wall, or axillary, internal mammary, or supraclavicular nodes. The median follow-up duration was 84.0 months for case patients and 92.9 months for control patients. Results Patients with heterogeneously dense (50–75% density) and extremely dense (>75% density) breasts had an increased risk of locoregional recurrence (hazard ratios 3.1 and 5.7, 95% confidence intervals 1.1–9.8 and 1.2–34.9, p = 0.043 and 0.048, respectively) than did women with less dense breasts. Positive margins after surgery also increased the risk of locoregional recurrence (hazard ratio 3.3, 95% confidence interval 1.3–8.3, p = 0.010). Multivariate analysis that included dense breasts (>50% density), positive margin, no adjuvant radiotherapy, and no adjuvant chemotherapy revealed that dense breasts were significant factors for predicting locoregional recurrence risk (hazard ratio 3.6, 95% confidence interval 1.2–11.1, p = 0.025). Conclusions Our results demonstrate that dense breast tissue (>50% density) increased the risk of locoregional recurrence after modified radical mastectomy in patients with invasive breast cancer. Additional prospective studies are necessary to validate these findings. Trial registration The study is retrospectively registered with ClinicalTrials.gov, number NCT02771665 , on May 11, 2016.