Background Severe acute respiratory illness (SARI) is an important cause of mortality in young children, especially in children living with HIV infection. Disparities in SARI death in children aged <5 years exist in urban and rural areas. Objective To compare the factors associated with in-hospital death among children aged <5 years hospitalized with SARI in an urban vs. a rural setting in South Africa from 2009–2013. Methods Data were collected from hospitalized children with SARI in one urban and two rural sentinel surveillance hospitals. Nasopharyngeal aspirates were tested for ten respiratory viruses and blood for pneumococcal DNA using polymerase chain reaction. We used multivariable logistic regression to identify patient and clinical characteristics associated with in-hospital death. Results From 2009 through 2013, 5,297 children aged <5 years with SARI-associated hospital admission were enrolled; 3,811 (72%) in the urban and 1,486 (28%) in the rural hospitals. In-hospital case-fatality proportion (CFP) was higher in the rural hospitals (6.9%) than the urban hospital (1.3%, p<0.001), and among HIV-infected than the HIV-uninfected children (9.6% vs. 1.6%, p<0.001). In the urban hospital, HIV infection (odds ratio (OR):11.4, 95% confidence interval (CI):5.4–24.1) and presence of any other underlying illness (OR: 3.0, 95% CI: 1.0–9.2) were the only factors independently associated with death. In the rural hospitals, HIV infection (OR: 4.1, 95% CI: 2.3–7.1) and age <1 year (OR: 3.7, 95% CI: 1.9–7.2) were independently associated with death, whereas duration of hospitalization ≥5 days (OR: 0.5, 95% CI: 0.3–0.8) and any respiratory virus detection (OR: 0.4, 95% CI: 0.3–0.8) were negatively associated with death. Conclusion We found that the case-fatality proportion was substantially higher among children admitted to rural hospitals and HIV infected children with SARI in South Africa. While efforts to prevent and treat HIV infections in children may reduce SARI deaths, further efforts to address health care inequality in rural populations are needed.