(1) Background—The five-year overall survival (OS) of muscle-invasive bladder cancer (MIBC) with neoadjuvant chemotherapy and cystectomy is around 50%. There is no validated biomarker to guide the treatment decision. We investigated whether the Immunoscore (IS) could predict the pathologic response to neoadjuvant chemotherapy and survival outcomes. (2) Methods—This retrospective study evaluated the IS in 117 patients treated using neoadjuvant chemotherapy for localized MIBC from six centers (France and Greece). Pre-treatment tumor samples were immunostained for CD3+ and CD8+ T cells and quantified to determine the IS. The results were associated with the response to neoadjuvant chemotherapy, time to recurrence (TTR), and OS. (3) Results—Low (IS-0), intermediate (IS-1–2), and high (IS-3–4) ISs were observed in 36.5, 43.7, and 19.8% of the cohort, respectively. IS was positively associated with a pathologic complete response (pCR; p-value = 0.0096). A high IS was found in 35.7% of patients with a pCR, whereas it was found in 11.3% of patients without a pCR. A low IS was observed in 48.4% of patients with no pCR and in 21.4% of patients with a pCR. Low-, intermediate-, and high-IS patients had five-year recurrence-free rates of 37.2%, 36.5%, and 72.6%, respectively. In the multivariable analysis, a high IS was associated with a prolonged TTR (high vs. low: p = 0.0134) and OS (high vs. low: p = 0.011). (4) Conclusions—This study showed the significant prognostic and predictive roles of IS regarding localized MIBC.