Aims: This study investigates the rural–urban differences in infant mortality rates (IMRs) and the associated risk factors in Nigeria. Methods: The dataset from the 2013 Nigeria demographic and health survey (NDHS), disaggregated by rural–urban residence, was analyzed using complex samples statistics. A multivariable logistic regression analysis was computed to explore the adjusted relationship and identify risk factors for infant mortality. Results: In rural and urban Nigeria, IMRs were 70 and 49 deaths per 1000 live births, respectively. Risk factors in rural residence were past maternal marital union (adjusted odds ratio (AOR): 1.625, p = 0.020), small birth size (AOR: 1.550, p < 0.001), birth interval <24 months (AOR: 2.057, p < 0.001), residence in North-East (AOR: 1.346, p = 0.038) and North-West (AOR: 1.653, p < 0.001) regions, and cesarean delivery (AOR: 2.922, p = 0.001). Risk factors in urban residence were poor wealth index (AOR: 2.292, p < 0.001), small birth size (AOR: 2.276, p < 0.001), male gender (AOR: 1.416, p = 0.022), birth interval <24 months (AOR: 1.605, p = 0.002), maternal obesity (AOR: 1.641, p = 0.008), and cesarean delivery (AOR: 1.947, p = 0.032). Conclusions: Infants in rural residence had higher rates of mortality than their urban counterparts and disparities in risk factors exist between the residences.