Luminal breast cancer (BC) is associated with less immune activation, and the significance of stromal lymphocytic infiltration (sTIL) is more uncertain than in other BC subtypes. The aim of this study was to investigate the predictive and prognostic value of sTIL in early luminal BC. The study was performed with an observational design in a prospective cohort of 345 patients with predominantly high-risk luminal (hormone receptor positive, HER2 negative) BC and with luminal B features (n = 286), in which the presence of sTIL was analyzed with validated methods. Median sTIL infiltration was 5% (Q1–Q3 range (IQR), 0–10). We found that sTIL were associated with characteristics of higher biological and clinical aggressiveness (tumor and lymph node proliferation and stage, among others) and that the percentage of sTIL was predictive of pathologic complete response in patients treated with neoadjuvant chemotherapy (OR: 1.05, 95%CI 1.02–1.09, p < 0.001). The inclusion of sTIL (any level of lymphocytic infiltration: sTIL > 0%) in Cox regression multivariable prognostic models was associated with a shorter relapse-free interval (HR: 4.85, 95%CI 1.33–17.65, p = 0.016) and significantly improved its performance. The prognostic impact of sTIL was independent of other clinical and pathological variables and was mainly driven by its relevance in luminal B BC.