Quinolizidine alkaloids exhibit various forms of biological activity. A lot of them were found in the Leguminosae family, including Laburnum and Genista. The aim of the study was the optimization of a chromatographic system for the analysis of cytisine and N-methylcytisine in various plant extracts as well as an investigation of the cytotoxic activities of selected alkaloids and plant extracts obtained from Laburnum anagyroides, Laburnum anagyroides L. quercifolium, Laburnum alpinum, Laburnum watereri, Genista germanica, and Genista tinctoria against various cancer cell lines. The determination of investigated compounds was performed by High Performance Liquid Chromatography with Diode Array Detection (HPLC-DAD), while High Performance Liquid Chromatography coupled with Quadrupole Time-of-Flight–Mass Spectrometry (HPLC-QTOF-MS) was applied for the qualitative analysis of plant extracts. The retention, separation selectivity, peaks shape, and systems efficiency obtained for cytisine and N-methylcytisine in different chromatographic systems were compared. The application of columns with alkylbonded and phenyl stationary phases led to a very weak retention of cytisine and N-methylcytisine, even when the mobile phases containing only 5% of organic modifiers were used. The strongest retention was observed when hydrophilic interaction chromatography (HILIC) or especially when ion exchange chromatography (IEC) were applied. The most optimal system in terms of alkaloid retention, peak shape, and system efficiency containing an strong cation exchange (SCX) stationary phase and a mobile phase consisted of 25% acetonitrile and formic buffer at pH 4.0 was applied for investigating alkaloids analysis in plant extracts. Cytotoxic properties of the investigated plant extracts as well as cytisine and N-methylcytisine were examined using human tongue squamous carcinoma cells (SCC-25), human pharyngeal squamous carcinoma cells (FaDu), human triple-negative breast adenocarcinoma cell line (MDA-MB-231), and human breast adenocarcinoma cell line (MCF-7). The highest cytotoxic activity against FaDu, MCF-7, and MDA-MB cancer cell lines was observed after applying the Genista germanica leaves extract. In contrast, the highest cytotoxic activity against SCC-25 cell line was obtained after treating with the seed extract of Laburnum watereri. The investigated plant extracts exhibit significant cytotoxicity against the tested human cancer cell lines and seem to be promising for further research on its anticancer activity.