Abstract Although tumor-infiltrating T cells hold a beneficial prognostic role in colorectal cancer, other lymphocytic populations are less characterized. We developed a multiplexed immunofluorescence assay coupled with digital image analysis and machine learning to identify natural killer (NK) cells (NCAM1+CD3−), natural killer T-like (NKT-like) cells (NCAM1+CD3+), and T cells (NCAM1−CD3+) within the PTPRC+ (CD45+) cell population and to measure their granzyme B (GZMB; cytotoxicity marker) and FCGR3A (CD16a; NK-cell maturity marker) expression. We evaluated immune cell densities and spatial configuration in 907 incident colorectal carcinoma cases within two prospective cohort studies. We found that T cells were approximately 100 times more abundant than NK and NKT-like cells. Overall, NK cells showed high GZMB expression and were located closer to tumor cells than T and NKT-like cells. In T and NKT-like cells, GZMB expression was enriched in cells in closer proximity to tumor cells. Higher densities of both T and NKT-like cells associated with longer cancer-specific survival, independent of potential confounders (Ptrend < 0.0007). Higher stromal GZMB+ and FCGR3A+ NK-cell densities associated with longer cancer-specific survival (Ptrend < 0.003). For T and NKT-like cells, greater proximity to tumor cells associated with longer cancer-specific survival (Ptrend < 0.0001). These findings indicate that cytotoxic NCAM1+CD3−GZMB+ NK cells and NCAM1+CD3+ NKT-like cells are relatively rare lymphocytic populations within the colorectal cancer microenvironment and show distinct spatial configuration and associations with patient outcome. The results highlight the utility of a quantitative multimarker assay for in situ, single-cell immune biomarker evaluation and underscore the importance of spatial context for tumor microenvironment characterization.