Colorectal cancer is a notorious disease, with almost half of the patients succumbing to the disease. The prevalence and incidence rates of colorectal cancer are increasing in many parts of the world, highlighting the need to discover new biomarkers for diagnosis and therapy. Caldesmon (CaD), an actin-binding protein that plays a significant role in controlling cell motility, has emerged as a promising biomarker. The CALD1 gene encodes CaD as multiple transcripts that mainly encode two protein isoforms: High-molecular-weight (h-CaD), expressed in smooth muscle, and low-molecular-weight (l-CaD), expressed in nonsmooth muscle cells. Most studies have suggested an oncogenic role of CaD in colorectal cancer, but the exact subcellular localization of the two CaD isoforms in tumor cells and stroma have not been clarified yet. Here, we analyzed tissue samples from 262 colorectal cancer patients by immunohistochemistry analysis using specific antibodies for l-CaD and h-CaD. The results showed elevated cytoplasmic expression levels of l-Cad in 187/262 (71.4%) cases. l-Cad was expressed at low levels in the normal colon mucosa and was also consistently expressed in the cancer-associated stroma of all cases, suggesting that it could play a role in modulating the tumor microenvironment. l-CaD expression in cancer cells was associated with preinvasive stages of cancer. Survival analysis indicated that patients with high l-CaD expression in tumor cells could respond poorly to selective chemotherapeutic 5FU, but not combination chemotherapy. h-CaD was expressed in colonic and vascular smooth muscle cells as expected and to a lesser extent in the tumor-associated stroma, but it was not expressed in the cancer cells or normal colon mucosal epithelial cells. Collectively, these data clarify how the expression patterns of CaD isoforms in colorectal cancer can have applications in the management of colorectal cancer patients.