Abstract Background Acute Respiratory Infections (ARIs) account for more than 6% of the worldwide disease burden in children under the age of five, with the majority occurring in Sub-Saharan Africa. Rural children are more vulnerable to and disproportionately affected by ARIs. As a result, we examined the rural–urban disparity in the prevalence of ARI symptoms and associated factors among children under the age of five in Sub-Saharan Africa. Methods We used the most recent Demographic and Health Survey (DHS) data from 36 countries in Sub-Saharan Africa. The study included 199,130 weighted samples in total. To identify variables associated with ARIs symptoms, a multilevel binary logistic regression model was fitted. The Adjusted Odds Ratio (AOR) with a 95% CI was used to determine the statistical significance and strength of the association. To explain the rural–urban disparity in ARI prevalence, a logit-based multivariate decomposition analysis was used. Results Being female, ever breastfeeding, belonging to a poorer, better wealth status, and having better maternal educational status were significantly associated with lower odds of ARIs among under-five children. Whereas, small size or large size at birth, not taking vitamin A supplementation, being severely underweight, having diarrhea, didn’t have media exposure, never had the vaccination, being aged 36–47 months, and being aged 48–59 months were significantly associated with higher odds of ARIs among under-five children. The multivariate decomposition analysis revealed that the difference in characteristics (endowment) across residences explained 64.7% of the overall rural–urban difference in the prevalence of ARIs, while the difference in the effect of characteristics (change in coefficient) explained 35.3%. Conclusion This study found that rural children were highly affected by ARIs in SSA. To reduce the excess ARIs in rural children, public health interventions aimed at impoverished households, home births, and unvaccinated and malnourished children are crucial.