Background: Tumor-infiltrating lymphocytes are associated with a better prognosis in early triple-negative breast cancer (TNBC). These cells can be enumerated in situ by the “Immunoscore Clinical Research” (ISCR). The original Immunoscore® is a prognostic tool that categorizes the densities of CD3+ and CD8+ cells in both the invasive margin (IM) and center of the tumor (CT) in localized colon cancer, yielding a five-tiered classification (0–4). We evaluated the prognostic potential of ISCR and pathological complete response (pCR) following neoadjuvant chemotherapy (NACT). Methods: The cohort included 53 TNBC, 32 luminal BC, and 18 HER2-positive BC patients undergoing NACT. Pre-treatment tumor biopsies were immune-stained for CD3+ and CD8+ T-cell markers. Quantitative analysis of these cells in different tumor locations was performed using computer-assisted image analysis. Results: The pCR rate was 44%. Univariate analysis showed that primary tumor size, estrogen-receptor negative, progesterone-receptor negative, luminal vs. HER2-positive vs. TNBC, high Ki-67, high densities (cells/mm2) of CD3 CT, CD8+ CT, CD3+ IM, and CD8+ IM cells were associated with a high pCR. ISCR was associated with pCR following NACT. A multivariate model consisting of ISCR and the significant variables from the univariate analysis showed a significant trend for ISCR; however, the low sample size did not provide enough power for the model to be included in this study. Conclusions: These results revealed a significant prognostic role for the spatial distributions of the CD3+, and CD8+ lymphocytes, as well as the ISCR in relation to pCR following NACT.