Abstract The densities of CD3+ and CD8+ tumor-infiltrating lymphocytes (TILs), combined with tumor–node–metastasis (TNM) staging, have prognostic value for patients with nonmetastatic colorectal cancer. We compared the prognostic value of CD3+ and FoxP3+ TILs at the invasive front, TNM classifiers, and microsatellite (MS) status in a trial set of patients with stage II and III colorectal cancer (n = 413), by recursive partitioning with a classification and regression tree (CART). Significant prognostic factors and interactions were reassessed by logistic regression and Cox proportional-hazards modeling in the trial and a validation set (n = 215) of patients with stage II colorectal cancer. In the trial set, CART indicated that TIL numbers were of value only in predicting recurrence risk for stage II cancers, where low densities of FoxP3+ TILs ranked first and low densities of CD3+ TILs further stratifying risk. Multivariate analysis showed that TILs interacted with tumor stage (FoxP3+, P = 0.06; CD3+, P = 0.02) and MS instability (MSI; FoxP3+; P = 0.02). In stage II MS-stable cancers, concomitant low densities of both FoxP3+ and CD3+ TILs identified patients with the highest progression risk in the trial [HR 7.24; 95% confidence interval (CI), 3.41–15.4; P < 0.001] and the validation (HR 15.16; 95% CI, 3.43–66.9; P < 0.001) sets. FoxP3+ and CD3+ TIL load in colorectal cancer was more informative than other prognostic factors before the cancer progressed to lymph nodes. This prognostic information about TILs, including FoxP3+ cells, suggests that randomized controlled trials might be refined to include interactions between TNM status, molecular classifiers, and postsurgical treatments.