Background: In cardiopulmonary exercise testing (CPET), oxygen uptake (V’O2) is calculated using the product of minute ventilation (V’E) and the difference between inspiratory and expiratory O2 concentrations (ΔFO2). However, little is known about the response of ΔFO2 to pulmonary rehabilitation (PR). The aim of the present study was (1) to investigate whether PR increases peak V’O2, based on whether ΔFO2 or V’E at peak exercise increase after PR, and (2) to investigate whether an improvement in ΔFO2 correlates with an improvement in ventilatory efficiency. Methods: A total of 38 patients with severe and very severe COPD, whose PR responses were evaluated by CPET, were retrospectively analyzed. Results: After PR, peak V’O2 was increased in 14 patients. The difference in ΔFO2 at peak exercise following PR correlated with the difference in peak V’O2 (r = 0.4884, p = 0.0019), the difference in V’E/V’CO2-nadir (r = −0.7057, p < 0.0001), and the difference in V’E–V’CO2 slope (r = −0.4578, p = 0.0039), but it did not correlate with the difference in peak V’E. Conclusions: The increased O2 extraction following PR correlated with improved exercise tolerance and ventilatory efficiency. In advanced COPD patients, a new strategy for improving O2 extraction ability might be effective in those in whom ventilatory ability can be only minimally increased.