In this observational prospective multicenter study conducted between October 2016 and October 2018, we tested the hypothesis that the use of prehospital non-invasive ventilation (phNIV) to treat patients with acute respiratory insufficiency (ARI) caused by severe acute exacerbations of chronic obstructive pulmonary disease (AECOPD) and acute cardiopulmonary oedema (ACPE) is effective, time-efficient and safe. The data were collected at four different physician response units and three admitting hospitals in a German EMS system. Patients with respiratory failure due to acute exacerbation of chronic obstructive pulmonary disease and acute cardiopulmonary oedema were enrolled. A total of 545 patients were eligible for the final analysis. Patients were treated with oxygen supplementation, non-invasive ventilation or invasive mechanical ventilation. The primary outcomes were defined as changes in the clinical parameters and the in-hospital course. The secondary outcomes included time efficiency, peri-interventional complications, treatment failure rate, and side-effects. Oxygenation under phNIV improved equally to endotracheal intubation (ETI), and more effectively in comparison to standard oxygen therapy (SOT) (paO2 SOT vs. non-invasive ventilation (NIV) vs. ETI: 82 mmHg vs. 125 mmHg vs. 135 mmHg, p-value SOT vs. NIV < 0.0001). In a matched subgroup analysis phNIV was accompanied by a reduced time of mechanical ventilation (phNIV: 1.8 d vs. ETI: 4.2 d) and a shortened length of stay at the intensive care unit (3.4 d vs. 5.8 d). The data support the hypothesis that the treatment of severe AECOPD/ACPE-induced ARI using prehospital NIV is effective, time efficient and safe. Compared to ETI, a matched comparison supports the hypothesis that prehospital implementation of NIV may provide benefits for an in-hospital course.