Abstract Background Blacks tend to have a stronger inflammatory immune response than Whites. We hypothesized that racial differences in host immunity also manifest in the tumor microenvironment, constituting part of a distinct aggressive tumor biology underlying higher mortality in Black women. Methods Pathological and gene expression profiling approaches were used for characterizing infiltrating immune cells in breast tumor microenvironment from 1315 patients from the Women’s Circle of Health Study. Racial differences in tumor immune phenotypes were compared, with results validated in a publicly accessible dataset. Prognostic associations of immune phenotypes were assessed in 3 independent cohorts. Results We found marked and consistent differences in tumor immune responses between Black and White patients. Not only did tumors from Blacks display a stronger overall immune presence but also the composition and quality of immune infiltrates differed, regardless of tumor subtypes. Black patients had a stronger CD4+ and B-cell response, and further, a more exhausted CD8+ T-cell profile. A signature indicating a higher ratio of exhausted CD8+ T cells to total CD8+ T cells (ExCD8-r) was consistently associated with poorer survival, particularly among hormone receptor–positive patients. Among hormone receptor–negative patients, combinations of the absolute fraction of CD8+ T cells and ExCD8-r signature identified the CD8lowExCD8-rhigh subgroup, the most prevalent among Blacks, with the worst survival. Conclusions Our findings of a distinct exhausted CD8+ T-cell signature in Black breast cancer patients indicate an immunobiological basis for their more aggressive disease and a rationale for the use of immune checkpoint inhibitors targeting the exhaustion phenotype.