Background: Rectal cancer is a malignant neoplasm of the large intestine resulting from the uncontrolled proliferation of the rectal tract. Predicting the pathologic response of neoadjuvant chemoradiotherapy at an MRI primary staging scan in patients affected by locally advanced rectal cancer (LARC) could lead to significant improvement in the survival and quality of life of the patients. In this study, the possibility of automatizing this estimation from a primary staging MRI scan, using a fully automated artificial intelligence-based model for the segmentation and consequent characterization of the tumor areas using radiomic features was evaluated. The TRG score was used to evaluate the clinical outcome. Methods: Forty-three patients under treatment in the IRCCS Sant’Orsola-Malpighi Polyclinic were retrospectively selected for the study; a U-Net model was trained for the automated segmentation of the tumor areas; the radiomic features were collected and used to predict the tumor regression grade (TRG) score. Results: The segmentation of tumor areas outperformed the state-of-the-art results in terms of the Dice score coefficient or was comparable to them but with the advantage of considering mucinous cases. Analysis of the radiomic features extracted from the lesion areas allowed us to predict the TRG score, with the results agreeing with the state-of-the-art results. Conclusions: The results obtained regarding TRG prediction using the proposed fully automated pipeline prove its possible usage as a viable decision support system for radiologists in clinical practice.