To determine the diagnostic test characteristics and inter-observer variation of pathology features for identifying high microsatellite instability (MSI-H) colorectal cancer (CRC). Six pathologists blindly evaluated 177 CRC for the presence of MSI-H associated pathology features. Inter-observer agreement was determined by using Kappa-statistics. In the first random 88/177 cases, mucinous carcinoma, tumor-infiltrating lymphocytes (TIL) and Crohns-like infiltrate (CLI) were the best discriminators between MSI-H and microsatellite stable CRC [OR 5.6 (95 % CI 1.7-19), 5.4 (1.8-17) and 3.5 (1.1-11), respectively], with high specificity (89-91 %). The sensitivities for MSI-H, however, were low (31-41 %). In addition, inter-observer agreement was moderate for TIL and CLI (kappa 0.38 and 0.48, respectively), but very good for mucinous carcinoma (kappa 0.86). Interpretation of overall histopathology as suggestive for MSI-H performed better than any individual feature; OR 15 (5.2-44), and area under the curve 0.79. However, inter-observer agreement was moderate (kappa 0.53). In the second set, TIL and CLI were scored according to updated scoring systems. Although both remained the best individual discriminators, test characteristics and inter-observer agreement did not improve. MSI-H pathology features have moderate accuracy for identifying MSI-H CRC, and are identified with moderate inter-observer agreement. These findings highlight the limitations of clinical strategies, such as the revised Bethesda guidelines, which incorporate the MSI-H associated pathology features in their strategy to identify persons with lynch syndrome.