The role of tumour-infiltrating inflammatory cells (TIICs) in the disease progression of hormone-receptor-positive breast cancer (HR+ BC) is largely unclear since it is generally regarded as the least immunogenic BC subtype. This study investigated the prognostic significance of CD1a+ dendritic cells, CD20+ B cells, CD45RO+ memory T cells and CD4+ T-helper cells in HR+ BC. One hundred and forty-six patients were treated for early stage, distant-metastases-free HR+ BC in an accelerated partial breast irradiation (APBI) phase II trial. Immunohistochemistry was used to double-stain two adjoining sets of tissue microarrays from pre-RT (radiotherapy) tumour resection samples for CD1a/CD20 and CD45RO/CD4. Cell densities of CD1a+, CD20+, CD45RO+ and CD4+ TIICs in the stromal and intraepithelial compartment were registered semiautomatically. High densities of CD20+ and CD4+ TIICs were strongly associated with reduced disease-free survival (DFS), while high stromal CD45RO+ TIIC densities were indicators of subsequent successful treatment. An immunoscore based on CD20+ and CD45RO+ TIIC densities identified three different risk groups (p < 0.001). Thus, contrary to current assumptions, intratumoural immune cell composition might be an important prognostic indicator and a possible contributing factor in the outcome of HR+ BC and should be the subject of further research. Specifically, B-cell infiltration entailed an increased relapse rate and could play an important role in disease progression.