Constitutive- and immunoproteasomes are part of the ubiquitin–proteasome system (UPS), which is responsible for the protein homeostasis. Selective inhibition of the immunoproteasome offers opportunities for the treatment of numerous diseases, including inflammation, autoimmune diseases, and hematologic malignancies. Although several inhibitors have been reported, selective nonpeptidic inhibitors are sparse. Here, we describe two series of compounds that target both proteasomes. First, benzoxazole-2-carbonitriles as fragment-sized covalent immunoproteasome inhibitors are reported. Systematic substituent scans around the fragment core of benzoxazole-2-carbonitrile led to compounds with single digit micromolar inhibition of the β5i subunit. Experimental and computational reactivity studies revealed that the substituents do not affect the covalent reactivity of the carbonitrile warhead, but mainly influence the non-covalent recognition. Considering the small size of the inhibitors, this finding emphasizes the importance of the non-covalent recognition step in the covalent mechanism of action. As a follow-up series, bidentate inhibitors are disclosed, in which electrophilic heterocyclic fragments, i.e., 2-vinylthiazole, benzoxazole-2-carbonitrile, and benzimidazole-2-carbonitrile were linked to threonine-targeting (R)-boroleucine moieties. These compounds were designed to bind both the Thr1 and β5i-subunit-specific residue Cys48. However, inhibitory activities against (immuno)proteasome subunits showed that bidentate compounds inhibit the β5, β5i, β1, and β1i subunits with submicromolar to low-micromolar IC50 values. Inhibitory assays against unrelated enzymes showed that compounds from both series are selective for proteasomes. The presented nonpeptidic and covalent derivatives are suitable hit compounds for the development of either β5i-selective immunoproteasome inhibitors or compounds targeting multiple subunits of both proteasomes.