Abstract Background Cancer survivors who develop breast cancer as a second malignancy (BCa-2) are common. Yet, little is known about the prognosis of BCa-2 compared to first primary breast cancer (BCa-1). Methods Using the Surveillance, Epidemiology, and End Results database, we conducted a population-based cohort study including 883,881 patients with BCa-1 and 36,313 patients with BCa-2 during 1990–2015. Compared with patients with BCa-1, we calculated hazard ratios (HRs) of breast cancer-specific mortality among patients with BCa-2, using multivariable Cox regression. Results During the follow-up (median 5.5 years), 114,964 and 3829 breast cancer-specific deaths were identified among BCa-1 and BCa-2 patients, respectively. Patients with BCa-2 had more favorable tumor characteristics and received less intensive treatment e.g., surgery and chemo−/radio-therapy, compared to patients with BCa-1. When adjusting for demographic factors, patients with BCa-2 were at similar risk of breast cancer-specific mortality (HR 1.00, 95% CI 0.97–1.03) compared to patients with BCa-1. However, when additionally controlling for tumor characteristics and treatment modes, BCa-2 patients were at an increased risk of breast cancer-specific mortality (HR 1.11, 95% CI 1.08–1.15). The risk elevation was particularly greater when the first malignancy was lung, bladder, ovarian or blood malignancy (HRs 1.16–1.85), or when the first malignancy was treated with chemotherapy and radiotherapy (HR 1.44, 95% CI 1.28–1.63). Conclusions Overall, patients with BCa-2 have worse breast cancer-specific survival, compared with their BCa-1 counterparts, although the risk elevation is mild. High-risk subgroups based on first malignancy’s characteristics may be considered for active clinical management.