Background: Immunomodulating therapies, which are commonly used in patients with Crohn’s disease (CD) and ulcerative colitis (UC), have been linked to an increased risk of contracting opportunistic infectious diseases, the majority of which are preventable through vaccination. Nonetheless, vaccination rates in these patients are suboptimal, and frequently lower than in the general population. The COVID-19 immunization schedule provided a new scenario for investigating vaccine acceptance in patients with inflammatory bowel disease (IBD), with uncertainty and concerns emerging and the number of subjects receiving the third and fourth doses of the vaccine gradually decreasing. This study investigated IBD patients’ attitudes towards previous COVID-19 vaccine programs and identified the factors that influence their adherence. It considered demographic and disease-related factors as well as the role of gastroenterologists and primary care physicians (PCPs). Methods: Data were collected through a self-completed questionnaire administered to all adult IBD patients (age > 18) who visited the Gastroenterology, Hepatology, and Nutrition division at the University of L’Aquila (Italy) for a regular follow-up between November 2021 and December 2022. Non-IBD gastroenterological outpatients who visited during the same period were included as a control group. Results: A total of 178 patients were included in the analysis. The IBD group consisted of 77 patients, 48.1% with CD and 51.9% with UC; the mean age was 49.5 years and 51.9% were female. Overall, 94.8% of IBD patients had undergone at least one vaccine dose and 79.2% had received two doses, versus 8% of the control group (p < 0.0001). A total of 84.4% of IBD patients reported their propensity towards COVID-19 vaccination, with an average agreement score significantly higher than the controls (p = 0.0044). The trust of IBD patients in the effectiveness of the COVID-19 vaccine (p < 0.0001) and its role in hastening pandemic resolution (p < 0.0001) is strongly related to motivation and propensity. Concerns about the safety of the COVID-19 vaccine in IBD (p = 0.0202) and fear of vaccine-induced flare-ups (p = 0.0192) were reported as the main barriers. No correlation was found between COVID-19 vaccine propensity and clinical features like the type of IBD, years of disease, activity, and ongoing treatment. Regarding the recommendations received from physicians to get vaccinated against COVID-19, IBD patients relied heavily on their gastroenterologists for advice, while the control group relied mainly on their PCPs. Conclusions: The overall positive attitude towards vaccinations reported in our study was better than that observed for other vaccines. The relationship of trust with the gastroenterologist should be used to boost vaccination against other preventable diseases in IBD patients. Our findings add information on the factors influencing vaccine propensity, which can be used to improve current vaccination strategies.