Abstract Incidence and survival of breast cancer, the most common cancer among women, have been increasing, leaving survivors at risk of aging-related health conditions. In this matched cohort study, we examined frailty risk with the Hospital Frailty Risk Score among breast cancer survivors (n = 34,900) and age-matched comparison subjects (n = 290,063). Women born in 1935–1975, registered in the Swedish Total Population Register (1991–2015), were eligible for inclusion. Survivors had a first breast cancer diagnosis in 1991–2005 and survived ≥5 years after initial diagnosis. Death date was determined by linkage to the National Cause of Death Registry (through 2015). Cancer survivorship was weakly associated with frailty (subdistribution hazard ratio (SHR) = 1.04, 95% confidence interval (CI): 1.00, 1.07). In age-stratified models, those diagnosed at younger ages (<50 years) had higher risk of frailty (SHR = 1.12, 95% CI: 1.00, 1.24) than those diagnosed at ages 50–65 (SHR = 1.03, 95% CI: 0.98, 1.07) or >65 (SHR = 1.09, 95% CI: 1.02, 1.17) years. Additionally, there was increased risk of frailty for diagnoses in 2000 or later (SHR = 1.15, 95% CI: 1.09, 1.21) compared with before 2000 (SHR = 0.97, 95% CI: 0.93, 1.17). This supports work from smaller samples showing that breast cancer survivors have increased frailty risk, particularly when diagnosed at younger ages.