Protein phosphatase 5 (PP5) is an evolutionary conserved serine/threonine phosphatase. Its dephosphorylation activity modulates a diverse set of cellular factors including protein kinases and the microtubule-associated tau protein involved in neurodegenerative disorders. It is auto-regulated by its heat-shock protein (Hsp90)-interacting tetratricopeptide repeat (TPR) domain and its C-terminal α-helix. In the present study, we report the identification of five specific PP5 activators [PP5 small-molecule activators (P5SAs)] that enhance the phosphatase activity up to 8-fold. The compounds are allosteric modulators accelerating efficiently the turnover rate of PP5, but do barely affect substrate binding or the interaction between PP5 and the chaperone Hsp90. Enzymatic studies imply that the compounds bind to the phosphatase domain of PP5. For the most promising compound crystallographic comparisons of the apo PP5 and the PP5–P5SA-2 complex indicate a relaxation of the auto-inhibited state of PP5. Residual electron density and mutation analyses in PP5 suggest activator binding to a pocket in the phosphatase/TPR domain interface, which may exert regulatory functions. These compounds thus may expose regulatory mechanisms in the PP5 enzyme and serve to develop optimized activators based on these scaffolds.