Solutions to meet growing food requirements in a world of limited suitable land and degrading environment focus mainly on increasing crop yields, particularly in poorly performing regions, and reducing animal product consumption. Increasing yields could alleviate land requirements, but imposing higher soil nutrient withdrawals and in most cases larger fertilizer inputs. Lowering animal product consumption favors a more efficient use of land as well as soil and fertilizer nutrients; yet actual saving may largely depend on which crops and how much fertilizer are used to feed livestock versus people. We show, with a global analysis, how the choice of cultivated plant species used to feed people and livestock influences global food production as well as soil nutrient withdrawals and fertilizer additions. The 3 to 15-fold differences in soil nutrient withdrawals per unit of energy or protein produced that we report across major crops explain how composition shifts over the last 20 years have reduced N, maintained P and increased K harvest withdrawals from soils while contributing to increasing dietary energy, protein and, particularly, vegetable fat outputs. Being highly variable across crops, global fertilization rates do not relate to actual soil nutrient withdrawals, but to monetary values of harvested products. Future changes in crop composition could contribute to achieve more sustainable food systems, optimizing land and fertilizer use.