Over 30 compounds, including para-, meta-, and ortho-phenylenediboronic acids, ortho-substituted phenylboronic acids, benzenetriboronic acids, di- and triboronated thiophenes, and pyridine derivatives were investigated as potential β-lactamase inhibitors. The highest activity against KPC-type carbapenemases was found for ortho-phenylenediboronic acid 3a, which at the concentration of 8/4 mg/L reduced carbapenems’ MICs up to 16/8-fold, respectively. Checkerboard assays revealed strong synergy between carbapenems and 3a with the fractional inhibitory concentrations indices of 0.1–0.32. The nitrocefin hydrolysis test and the whole cell assay with E. coli DH5α transformant carrying blaKPC-3 proved KPC enzyme being its molecular target. para-Phenylenediboronic acids efficiently potentiated carbapenems against KPC-producers and ceftazidime against AmpC-producers, whereas meta-phenylenediboronic acids enhanced only ceftazidime activity against the latter ones. Finally, the statistical analysis confirmed that ortho-phenylenediboronic acids act synergistically with carbapenems significantly stronger than other groups. Since the obtained phenylenediboronic compounds are not toxic to MRC-5 human fibroblasts at the tested concentrations, they can be considered promising scaffolds for the future development of novel KPC/AmpC inhibitors. The complexation of KPC-2 with the most representative isomeric phenylenediboronic acids 1a, 2a, and 3a was modeled by quantum mechanics/molecular mechanics calculations. Compound 3a reached the most effective configuration enabling covalent binding to the catalytic Ser70 residue.