The COVID-19 outbreak emerged as an ongoing crisis at the beginning of the year 2020. Its horrific manifestation at the community level significantly affects various dimensions of the quality of life (QoL) of all individuals. The study aimed to examine some of the predictors of the QoL during the first wave of the COVID-19 pandemic in Saudi Arabia. A cross-sectional online survey questionnaire was used to gather data on the participants’ sociodemographic backgrounds, physical health status, psychological reactions, and QoL. We adapted 12 items from the World Health Organization Quality of Life Instruments (WHOQOL-BREF) to assess the QoL. The Depression, Anxiety and Stress Scale–21 (DASS-21) was used to assess depression, anxiety, and stress. The median and inter-quartile range were used to describe the QoL scores. A multinomial regression analysis was computed between QoL score quartiles and associated factors, and the statistical significance was set at p < 0.05. The results of the multinomial regression analysis demonstrated that males (OR = 1.96; 95% CI = 1.31–2.94); participants aged 26 to 35 years (OR = 5.1; 95% CI = 1.33–19.37); non-Saudi participants (OR = 1.69; 95% CI = 1.06–2.57); individuals with chronic diseases (OR = 2.15; 95% CI = 1.33–3.48); those who lost their job (OR = 2.18; 95% CI = 1.04–4.57); and those with depression (OR = 5.70; 95% CI = 3.59–9.05), anxiety (OR = 5.47; 95% CI = 3.38–8.84), and stress (OR = 6.55; 95% CI = 4.01–10.70) were more likely to be in the first quartile of the QoL scores. While the full model predicting the total QoL score was statistically significant (R2 = 0.962, F (750, 753) = 16,705.4, p < 0.001), the three QoL dimensions explained 0.643, 0.283, and 0.036 of the variability in environmental, social, and religious/spiritual dimensions, respectively. The COVID-19 pandemic has significantly influenced various aspects of individuals’ QoL, as well as their physical and psychological health. Community-based interventions are needed to mitigate the pandemic’s negative effects and enhance the health and QoL of the general population.