Anneslea fragrans Wall., popularly known as “Pangpo tea”, is an edible, medicinal, and ornamental plant of the Family Theaceae. The leaves of A. fragrans were historically applied for the treatment of liver and intestinal inflammatory diseases in China. This study aimed to explore the hepatoprotective agents from A. fragrans leaves through hepatoprotective and anti-inflammatory assessment. The phytochemical investigation of the leaves of A. fragrans resulted in the isolation and identification of a total of 18 chemical compounds, including triterpenoids, aliphatic alcohol, dihydrochalcones, chalcones, flavanols, phenolic glycoside, and lignans. Compounds 1–2, 4–6, 11–12, and 16–18 were identified from A. fragrans for the first time. Compounds 7 and 14 could significantly alleviate hepatocellular damage by decreasing the contents of aspartate aminotransferase (AST) and alanine aminotransferase (ALT) and inhibit the hepatocellular apoptosis in the HepG2 cells induced by N-acetyl-p-aminophenol (APAP). In addition, compounds 7 and 14 inhibited reactive oxygen species (ROS) and malondialdehyde (MDA) contents and increased the catalase (CAT) superoxide dismutase (SOD), and glutathione (GSH) levels for suppressing APAP-induced oxidative stress. Additionally, compounds 7, 13, and 14 also had significant anti-inflammatory effects by inhibiting interleukin-6 (IL-6), interleukin-1β (IL-1β), and tumor necrosis factor-α (TNF-α) productions on LPS-induced RAW246.7 cells.