Estrogen receptor (ER) status is important for breast cancer survival, it is however unclear how prognosis of contralateral breast cancer (CBC) is affected by ER-status of the two tumors. We conducted a large, population-based study of ER-status of both tumors in CBC patients and its influence on prognosis. The cohort consisted of all women diagnosed with CBC in Stockholm, Sweden during 1976-2005, with information on ER-status from medical records (N = 933). Prognosis was modeled as incidence rates of distant metastasis via Poisson regression. The proportion of CBCs with both cancers of the same ER-status was significantly larger than expected by chance. For synchronous (simultaneous) cancers the prognosis was significantly affected by the combined ER-status of both tumors (p = 0.01). Compared to unilateral breast cancer patients the incidence rate ratio (IRR) for patients with double ER-positive tumors was 1.25 (95 % CI: 0.88-1.76), for ER-discordant tumors 2.19 (95 % CI: 1.18-4.08) and for double ER-negative tumors 3.95 (95 % CI: 1.77-8.81). For metachronous (non-simultaneous) cancers, women with double ER-positive tumors had similarly bad prognosis (IRR = 2.95; 95 % CI: 2.39-3.64) as women with double ER-negative tumors (IRR = 2.88; 95 % CI: 1.83-4.52). Both shorter time span between first and second cancer and endocrine therapy for the first cancer further worsened prognosis of women with double ER-positive metachronous CBC. For synchronous CBC patients, ER-pattern of both tumors is an important prognosticator, while among metachronous CBC patients, double ER-positive tumors confer equally bad prognosis as double ER-negative cancers. Our results indicate that this might be due to endocrine therapy resistance.