Liming (L) is a common practice to mitigate soil acidification and enhance soil quality and crop productivity. However, in acidic soil, it is not clear how long-term application of lime and nitrogen (N)-based fertilizer affects soil chemical properties, the wheat and maize grain yields (GY), and N-use efficiency (NUE). Thus, to investigate the effects of N-based fertilizations without L (−L) and with L (+L) on wheat and maize GY and NUE through their effects on soil chemical properties, we analyzed a 28-year field experiment in acidic soil under a wheat-maize system in South China. The analysis was carried out between 1991 and 2010 (before L) and between 2011 and 2018 (after L). We categorized the treatments into (1) no fertilizer (C); nitrogen (N); N and phosphorus (NP); N and potassium (NK); N, P and K (NPK); and NPKCR, NPK and crops residue (CR) applications (NPKCR), before L; and (2) C; N−L; N+L; NP−L; NP+L; NK−L; NK+L; NPK−L; NPK+L; NPKCR−L and NPKCR+L, after L. The effects of long-term fertilization resulted in lower soil pH by 15%, soil available K (AK) by 19%, POlsen by 6%, NO3−-N by 15%, soil organic matter (SOM) by 16%, total N by 16%, and C:N ratio by 13% in −L soil than in +L soil. However, the accumulation of NH4+-N was higher by 40% in −L soil than in +L soil. Wheat and maize GY, N recovery efficiency (REN), and N partial factor productivity (PEPN) were more adversely affected by 8-year fertilizations in −L compared with fertilizations before L and in +L primarily because of the significantly decreased soil pH. Conversely, improvements in wheat and maize yields, REN, and PFPN by 8-year fertilizations in +L were related to increasing soil pH, exchangeable base cations such as Ca2+, Mg2+, and the alleviated toxicity of Al3+. Overall, improvement of GY and NUE from the acidified soil in South China requires the long-term integrated use of fertilizer (NPK), retention of CR, and the +L (i.e., NPKCR+L).