The role of the nervous system as a contributor in the tumor microenvironment has been recognized in different cancer types, including colorectal cancer (CRC). The gastrointestinal tract is a highly innervated organ system, which is not only innervated by the autonomic nervous system, but also contains an extensive nervous system of its own; the enteric nervous system (ENS). The ENS is important for gut function and homeostasis by regulating processes such as fluid absorption, blood flow, and gut motility. Dysfunction of the ENS has been linked with multiple gastrointestinal diseases, such as Hirschsprung disease and inflammatory bowel disease, and even with neurodegenerative disorders. How the extrinsic and intrinsic innervation of the gut contributes to CRC is not fully understood, although a mutual relationship between cancer cells and nerves has been described. Nerves enhance cancer progression through the secretion of neurotransmitters and neuropeptides, and cancer cells are capable of stimulating nerve growth. This review summarizes and discusses the nervous system innervation of the gastrointestinal tract and how it can influence carcinogenesis, and vice versa. Lastly, the therapeutic potential of these novel insights is discussed.