Undernutrition and micronutrient malnutrition remain problems of significant magnitude in large parts of the developing world. Improved nutrition requires not only better access to food for poor population segments, but also higher dietary quality and diversity. Because many of the poor and undernourished people are smallholder farmers, diversifying production on these smallholder farms is widely perceived as a useful approach to improve dietary diversity. However, empirical evidence on the link between production and consumption diversity is scarce. Here, this issue is addressed with household-level data from Indonesia, Kenya, Ethiopia, and Malawi. Regression models show that on-farm production diversity is positively associated with dietary diversity in some situations, but not in all. When production diversity is already high, the association is not significant or even turns negative, because of foregone income benefits from specialization. Analysis of other factors reveals that market access has positive effects on dietary diversity, which are larger than those of increased production diversity. Market transactions also tend to reduce the role of farm diversity for household nutrition. These results suggest that increasing on-farm diversity is not always the most effective way to improve dietary diversity in smallholder households and should not be considered a goal in itself. Additional research is needed to better understand how agriculture and food systems can be made more nutrition-sensitive in particular situations.