(1) Background: Even though irritable bowel syndrome (IBS) is not life-threatening, it can have a significant impact on a patient's daily activities and quality of life. Studies indicate that IBS is more prevalent in women than in men. The mechanism, etiology, and signs of IBS are not well understood, but there is evidence that gender and age is a biological characteristic that affects these subjects. This influence on the quality of life of Saudi adults by gender and age has not previously been compared. (2) Methods: A snowball sampling techniques of 279 people in Qatif, Saudi Arabia, is surveyed online to assess health-related quality of life (HRQoL). The Short SF-36 was self-administered to assess HRQoL; logistic regression models, Mann-Whitney, and the Kruskal-Wallis were used. SPSS version 23.0 was used to store and analyze all data. (3) Results: The prevalence of IBS among Saudi adults was 17.6%. IBS has a statistically significant association with age and gender which is more common among older women. IBS participants have poorer HRQoL than the overall population, particularly in Physical Component Score (PCS). (4) Conclusions: IBS is more prevalent among Saudi adults in Qatif and has a significant negative impact on the HRQoL, particularly among elderly women.