Abstract Background Hepatoma is a common malignancy of the liver. The abnormal high expression of alpha-fetoprotein (AFP) is intimately associated with hepatoma progress, but the mechanism of transcriptional regulation and singularly activation of AFP gene in hepatoma is not clear. Methods The expression of transcription factor HBP1 and AFP and clinical significance were further analyzed in hepatoma tissues from the patients who received surgery or TACE and then monitored for relapse for up 10 years. HBP1-mediated transcriptional regulation of AFP was analyzed by Western blotting, Luciferase assay, Realtime-PCR, ChIP and EMSA. After verified the axis of HBP-AFP, its impact on hepatoma was measured by MTT, Transwell and FACS in hepatoma cells and by tumorigenesis in HBP1^−/− mice. Results The relative expressions of HBP1 and AFP correlated with survival and prognosis in hepatoma patients. HBP1 repressed the expression of AFP gene by directly binding to the AFP gene promoter. Hepatitis B Virus (HBV)-encoded protein HBx promoted malignancy in hepatoma cells through binding to HBP1 directly. Icaritin, an active ingredient of Chinese herb epimedium, inhibited malignancy in hepatoma cells through enhancing HBP1 transrepression of AFP. The repression of AFP by HBP1 attenuated AFP effect on PTEN, MMP9 and caspase-3, thus inhibited proliferation and migration, and induced apoptosis in hepatoma cells. The deregulation of AFP by HBP1 contributed to hepatoma progression in mice. Conclusions Our data clarify the mechanism of HBP1 in inhibiting the expression of AFP and its suppression in malignancy of hepatoma cells, providing a more comprehensive theoretical basis and potential solutions for the diagnosis and treatment of hepatoma.