Low vigor and early and abundant production are desirable traits for modern tree crops. In olive, most cultivars are too vigorous and cannot be successfully constrained in the small volume allowed by the straddle harvester used in the so-called superhigh-density (SHD) orchards. Only few cultivars appear to have sufficiently low vigor to be suitable for this system. These cultivars combine low vigor with earlier and higher yield. This study investigated the hypothesis that differences in vigor between Arbequina, a low vigor and the most commonly used cultivar in SHD orchards, and Frantoio, a highly vigorous cultivar not suitable for such orchards, are related to their differences in early bearing and consequent differences in dry matter partitioning into fruit. Young trees of both cultivars were deflowered either in 2014, 2015, both years, or neither one, resulting in a range of cumulative yields over the 2 years. Tree trunk cross-sectional area (TCSA) was measured at the beginning of each year. This was closely related to total tree mass, as assessed at the beginning and at the end of the experiment. Cumulative yield, in terms of fruit dry matter, was also assessed. TCSA increased less in fruiting trees in both years. As expected, when not deflowered, ‘Frantoio’ was less productive and more vigorous than ‘Arbequina’. However, there was no difference in TCSA increment when both cultivars were completely deflowered. TCSA increments were closely inversely related to yield across all treatments and cultivars (R² = 0.90). The regressions improved further when data from 2015 only were used (R² = 0.99). The results represent the first quantitative report showing that differences in vigor among cultivars can be completely explained in terms of different dry matter partitioning into fruit, supporting the hypothesis that early bearing is a major cause, rather than merely a consequence, of lower vigor in young ‘Arbequina’ trees. These results provide new understanding on vigor differences across cultivars, which will be useful for breeding and selection of new genotypes.