Abstract Background The left-sided and right-sided colon cancer (LCCs and RCCs, respectively) have unique molecular features and clinical heterogeneity. This study aimed to identify the characteristics of immune cell infiltration (ICI) subtypes for evaluating prognosis and therapeutic benefits. Methods The independent gene datasets, corresponding somatic mutation and clinical information were collected from The Cancer Genome Atlas and Gene Expression Omnibus. The ICI contents were evaluated by “ESTIMATE” and “CIBERSORT.” We performed two computational algorithms to identify the ICI landscape related to prognosis and found the unique infiltration characteristics. Next, principal component analysis was conducted to construct ICI score based on three ICI patterns. We analyzed the correlation between ICI score and tumor mutation burden (TMB), and stratified patients into prognostic-related high- and low- ICI score groups (HSG and LSG, respectively). The role of ICI scores in the prediction of therapeutic benefits was investigated by "pRRophetic" and verified by Immunophenoscores (IPS) (TCIA database) and an independent immunotherapy cohort (IMvigor210). The key genes were preliminary screened by weighted gene co-expression network analysis based on ICI scores. And they were further identified at various levels, including single cell, protein and immunotherapy response. The predictive ability of ICI score for prognosis was also verified in IMvigor210 cohort. Results The ICI features with a better prognosis were marked by high plasma cells, dendritic cells and mast cells, low memory CD4⁺ T cells, M0 macrophages, M1 macrophages, as well as M2 macrophages. A high ICI score was characterized by an increased TMB and genomic instability related signaling pathways. The prognosis, sensitivities of targeted inhibitors and immunotherapy, IPS and expression of immune checkpoints were significantly different in HSG and LSG. The genes identified by ICI scores and various levels included CA2 and TSPAN1. Conclusion The identification of ICI subtypes and ICI scores will help gain insights into the heterogeneity in LCC and RCC, and identify patients probably benefiting from treatments. ICI scores and the key genes could serve as an effective biomarker to predict prognosis and the sensitivity of immunotherapy.