The site-specific average yield is a well-known value to most producers, thus the inter-annual yield variability is the actual target value of any yield forecast. Using mechanistic models for in-season yield forecasts may help to optimize crop management decisions such as absolute fertilizer rates. However, a biased simulated yield can limit the potential benefits and is often a consequence of yield limiting factors like preceding crop, not considered by the model. This paper outlines a methodology which uses site-specific, historical weather records for a relative yield prognosis (Yrel) via yield projections and reference calculations. In order to obtain the yield forecast in absolute terms, Yrel is then multiplied with average observed yield. The key benefit of this design is that the bias is mostly cancelled out, thereby improving the model accuracy. The used crop–soil model HumeWheat was developed and parameterized on a broad experimental database including several modern wheat cultivars. We assume that the broad parameterization of key-processes allows the detection of inter-annual yield variability, even without genotype- or site-specific model calibrations. Our first aim was to evaluate the general applicability of this new approach of yield forecasting empirically at different phenological stages. The second aim was to evaluate the impact of different soil and weather conditions on forecast accuracy. Yield observations from several modern bread wheat cultivars (Triticum aestivum L.) were used to evaluate the method. For forecasts at the start of anthesis, the overall mean absolute error for yield (MAE, t ha−1 dry matter) was reduced by 0.24 (or in relative terms 27%) compared to the assumption of treatment-specific (site ∗ preceding crop) average yield. A subsequent simulation study with three different climate and varying water holding capacities water holding capacities reveals that a greater benefit can be expected for sites less stable in yield because of drought limitations.