Abstract Large-scale crop yield failures are increasingly associated with food price spikes and food insecurity and are a large source of income risk for farmers. While the evidence linking extreme weather to yield failures is clear, consensus on the broader set of weather drivers and conditions responsible for recent yield failures is lacking. We investigate this for the case of four major crops in Germany over the past 20 years using a combination of machine learning and process-based modelling. Our results confirm that years associated with widespread yield failures across crops were generally associated with severe drought, such as in 2018 and to a lesser extent 2003. However, for years with more localized yield failures and large differences in spatial patterns of yield failures between crops, no single driver or combination of drivers was identified. Relatively large residuals of unexplained variation likely indicate the importance of non-weather related factors, such as management (pest, weed and nutrient management and possible interactions with weather) explaining yield failures. Models to inform adaptation planning at farm, market or policy levels are here suggested to require consideration of cumulative resource capture and use, as well as effects of extreme events, the latter largely missing in process-based models. However, increasingly novel combinations of weather events under climate change may limit the extent to which data driven methods can replace process-based models in risk assessments.