Abstract Background Endometriosis is an estrogen-dependent inflammatory reproductive disease. Therefore, systematic estrogen depletion and anti-inflammatory drugs are the current treatment for endometriosis. However, current endometriosis treatments have low efficacy and cause adverse effects in endometriosis patients. Consequently, alternative endometriosis treatments targeting endometriosis-specific factors are in demand. In this context, ERβ was selected as a druggable target for endometriosis due to its critical role in progression. Therefore, selective targeting of ERβ without inhibiting ERα activity would be a new paradigm for endometriosis treatment to overcome the low efficacy and adverse effects of hormonal endometriosis therapy. Methods Cell-based ERβ and ERα activity assay systems were employed to define a selective ERβ-inhibiting chemical product from a library of natural products. A surgically induced endometriosis mouse model was used to determine whether an ERβ inhibitory drug suppressed endometriosis progression. Mice with endometriosis were randomly separated and then orally treated with vehicle or 25 mg/kg oleuropein (once a day for 21 days), an ERβ inhibitory drug. The volume of endometriotic lesions or luciferase activity of endometriotic lesions was examined to define the growth of ectopic lesions in mice with endometriosis. The metabolite and levels of metabolic enzymes of the liver and kidney were determined in the serum of female mice treated with vehicle and oleuropein (25 mg/kg, once a day for 21 days) to define the toxicity of oleuropein. The in vitro decidualization assay was conducted with normal human endometrial stromal cells and endometriotic stromal cells to determine whether oleuropein overcomes decidualization in endometriosis patients. The pregnancy rate and pup numbers of C57BL/6 J female mice with endometriosis treated with vehicle or oleuropein (n = 10/group) were determined after mating with male mice. The cytokine profile in endometriotic lesions treated with vehicle and oleuropein (25 mg/kg) was determined with a Mouse Cytokine Array Kit. Results Among natural products, oleuropein selectively inhibited ERβ but not ERα activity in vitro. Oleuropein treatment inhibited the nuclear localization of ERβ in human endometrial cells upon estradiol treatment. Oleuropein (25 mg/kg) treatment suppressed the growth of mouse (6.6-fold) and human (sixfold) ectopic lesions in mice with endometriosis compared to the vehicle by inhibiting proliferation and activating apoptosis in endometriotic lesions. Oleuropein treatment did not cause reproductive toxicity in female mice. Additionally, mice with endometriosis subjected to oleuropein treatment had a higher pregnancy rate (100%) than vehicle-treated mice (70%). Furthermore, oleuropein treatment partially recovered the decidualization impact of human endometriotic stromal cells from endometriotic lesions compared to the vehicle. Oleuropein-treated mice with endometriosis exhibited significantly lower levels of cytokines directly regulated by ERβ in ectopic lesions than vehicle-treated mice, illustrating the improvement in the hyperinflammatory state of mice with endometriosis. Conclusions Oleuropein is a promising and novel nutraceutical product for nonhormonal therapy of endometriosis because it selectively inhibits ERβ, but not ERα, to suppress endometriosis progression and improve the fertility of mice with endometriosis.