Abstract Objectives To assess the antiplasmodial activity of the ethanol extract of Xylopia sericea leaves, Annonaceae, often associated with antimalarial use and to perform a bioguided isolation of active compounds. Methods Dereplication of ethanol extract by the UPLC-DAD-ESI-MS/MS technique allowed the identification of the major constituents, isolation and identification of alkaloids. The antiplasmodial and cytotoxic activity of the extract, fractions and isolated compounds was evaluated against the chloroquine-resistant W2 strain Plasmodium falciparum and HepG2 cells, respectively. Key findings Ethanol extract showed high reduction of parasitemia as well as moderate cytotoxicity (86.5 ± 3.0% growth inhibition at 50 μg/ml and CC50 72.1 ± 5.1 μg/ml, respectively). A total of eight flavonoids were identified, and two aporphine alkaloids, anonaine and O-methylmoschatoline, were isolated. Anonaine disclosed significant antiplasmodial effect and moderate cytotoxicity (IC50 23.2 ± 2.7 μg/ml, CC50 38.3 ± 2.3 μg/ml, SI 1.6) while O-methylmoschatoline was not active against P. falciparum and showed a low cytotoxicity (33.5 ± 1.9% growth inhibition at 50 μg/ml, CC50 274.4 ± 0.5 μg/ml). Conclusions Characterization of Xylopia sericea leaves ethanol extract by UPLC-DAD-ESI-MS/MS as well as its antiplasmodial activity and the occurrence of anonaine and O-methylmoschatoline in this Xylopia species are reported by the first time.