The purpose of this study was to evaluate the association between four distinct histopathological features: (1) tumor infiltrating lymphocytes, (2) mucinous differentiation, (3) tumor-stroma ratio, plus (4) tumor budding and two gene expression-based classifiers—(1) consensus molecular subtypes (CMS) plus (2) colorectal cancer intrinsic subtypes (CRIS). All four histopathological features were retrospectively scored on hematoxylin and eosin sections of the most invasive part of the primary tumor in 218 stage II and III colon cancer patients from two independent cohorts (AMC-AJCC-90 and AC-ICAM). RNA-based CMS and CRIS assignments were independently obtained for all patients. Contingency tables were constructed and a χ2 test was used to test for statistical significance. Odds ratios with 95% confidence intervals were calculated. The presence of tumor infiltrating lymphocytes and a mucinous phenotype (>50% mucinous surface area) were strongly correlated with CMS1 (p < 0.001 and p = 0.008) and CRIS-A (p = 0.006 and p < 0.001). The presence of mucus (≥ 10%) was associated with CMS3: mucus was present in 64.1% of all CMS3 tumors (p < 0.001). Although a clear association between tumor-stroma ratio and CMS4 was established in this study (p = 0.006), still 32 out of 61 (52.5%) CMS4 tumors were scored as stroma-low, indicating that CMS4 tumors cannot be identified solely based on stromal content. Higher budding counts were seen in CMS4 and CRIS-B tumors (p = 0.045 and p = 0.046). No other associations of the measured parameters were seen for any of the other CRIS subtypes. Our analysis revealed clear associations between histopathologic features and CMS or CRIS subtypes. However, identification of distinct molecular subtypes solely based on histopathology proved to be infeasible. Combining both molecular and morphologic features could potentially improve patient stratification.