A lack of basic information on optimal nitrogen (N) management often results in over- or under-application of N fertilizer in small-scale intensive rice farming. Here, we present a new database of N input from a survey of 6611 small-scale rice farmers and rice yield in response to added N in 1177 experimental on-farm tests across eight agroecological subregions of China. This database enables us to evaluate N management by farmers and develop an optimal approach to regional N management. We also investigated grain yield, N application rate, and estimated greenhouse gas (GHG) emissions in comparison to N application and farming practices. Across all farmers, the average N application rate, weighted by the area of rice production in each subregion, was 210 kg ha−1 and ranged from 30 to 744 kg ha−1 across fields and from 131 to 316 kg ha−1 across regions. The regionally optimal N rate (RONR) determined from the experiments averaged 167 kg ha−1 and varied from 114 to 224 kg N ha−1 for the different regions. If these RONR were widely adopted in China, approximately 56% of farms would reduce their use of N fertilizer, and approximately 33% would increase their use of N fertilizer. As a result, grain yield would increase by 7.4% from 7.14 to 7.67 Mg ha−1, and the estimated GHG emissions would be reduced by 11.1% from 1390 to 1236 kg carbon dioxide (CO2) eq Mg−1 grain. These results suggest that to achieve the goals of improvement in regional yield and sustainable environmental development, regional N use should be optimized among N-poor and N-rich farms and regions in China.