Background: This study was conducted to provide an updated estimate of the prognostic power of C-X-C chemokine receptor type 4 (CXCR4) in colorectal cancer (CRC), and analyze modalities of evaluating and reporting its expression. Methods: A systematic review with meta-analysis was performed and described according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement. Studies were identified through PubMed and Google Scholar. The pooled hazard ratios (HRs) for overall survival (OS) or progression-free survival (PFS) with 95% confidence interval (CI) were estimated with the random-effect model. Results: Sixteen studies were selected covering a period from 2005 to 2020. An immunohistochemical evaluation of CXCR4 was performed in all studies. Only in three studies assessment of mRNA through RT–PCR was correlated with prognosis; in the remaining studies, the authors identified prognostic categories based on immunohistochemical expression. In pooled analyses, significant associations were found between positive or high or strong expression of CXCR4 and T stage ≥3 (P = 0.0001), and positive or high or strong expression of CXCR4 and left side primary tumor localization (P = 0.0186). The pooled HR for OS was 2.09 (95% CI: 1.30–2.88) in favor of high CXCR4 expression; for PFS, it was 1.42 (95% CI: 1.13–1.71) in favor of high CXCR4 expression. Conclusion: High CXCR4 expression is clearly associated with increased risk of death and progression in CRC. However, strong methodologic heterogeneity in CXCR4 assessment hinders direct translation into clinical practice; thus, a consensus to streamline detection and scoring of CXCR4 expression in CRC is indicated.