Increasing crop productivity while simultaneously reducing the environmental footprint of crop production is considered a major challenge for the coming decades. Even short episodes of heat stress can reduce crop yield considerably causing low resource use efficiency. Studies on the impact of heat stress on crop yields over larger regions generally rely on temperatures measured by standard weather stations at 2 m height. Canopy temperatures measured in this study in field plots of rye were up to 7 °C higher than air temperature measured at typical weather station height with the differences in temperatures controlled by soil moisture contents. Relationships between heat stress and grain number derived from controlled environment studies were only confirmed under field conditions when canopy temperature was used to calculate stress thermal time. By using hourly mean temperatures measured by 78 weather stations located across Germany for the period 1994–2009 it is estimated, that mean yield declines in wheat due to heat stress during flowering were 0.7% when temperatures are measured at 2 m height, but yield declines increase to 22% for temperatures measured at the ground. These results suggest that canopy temperature should be simulated or estimated to reduce uncertainty in assessing heat stress impacts on crop yield.