Background: Colorectal mixed adenoneuroendocrine carcinomas (MANECs) are clinically highly aggressive neoplasms. MANECs are composed of variable adenocarcinoma components combined with morphologically distinct neuroendocrine carcinoma components, which are confirmed by synaptophysin immunohistochemistry, the gold standard marker of a neuroendocrine differentiation. However, the biological behavior of adenocarcinomas that express synaptophysin but do not show a typical neuroendocrine morphology remains unclear. Methods: We investigated synaptophysin expression in 1002 conventional colorectal adenocarcinomas and correlated the results with clinicopathological characteristics and patient survival and compared the survival characteristics of synaptophysin expression groups to MANECs. Results: Synaptophysin expression in conventional colorectal adenocarcinomas was associated with a shortened disease-free survival (p = 0.037), but not with overall survival or disease-specific survival (DSS) in univariate analyses and without any survival impact in multivariate analyses. Patients with “true” MANECs, on the other hand, showed a significantly shorter survival than all conventional adenocarcinomas with or without synaptophysin expression in uni- and multivariate analyses (e.g., multivariate DSS: p < 0.001, HR: 5.20). Conclusions: Our study demonstrates that synaptophysin expression in conventional colorectal adenocarcinomas, in contrast to MANECs, is not associated with a significantly poorer clinical outcome when compared to adenocarcinomas without synaptophysin expression. Furthermore, our data suggest that conventional adenocarcinomas with a diffuse synaptophysin expression should not be classified as MANECs, also strongly arguing that synaptophysin testing should be reserved for carcinomas with an H&E morphology suggestive of a neuroendocrine differentiation.