Abstract Background Quantifying gametocyte densities in natural malaria infections is important to estimate malaria transmission potential. Two molecular methods (Pfs25 mRNA quantitative reverse transcriptase PCR (qRT-PCR) and Pfs25 mRNA quantitative nucleic acid sequence based amplification (QT-NASBA)) are commonly used to determine gametocyte densities in clinical and epidemiological studies and allow gametocyte detection at densities below the microscopic threshold for detection. Here, reproducibility of these measurements and the association between estimated gametocyte densities and mosquito infection rates were compared. Methods To quantify intra- and inter-assay variation of QT-NASBA and qRT-PCR, a series of experiments was performed using culture-derived mature Plasmodium falciparum gametocytes from three different parasite isolates (NF54, NF135, NF166). Pfs25 mRNA levels were also determined in samples from clinical trials in Mali and Burkina Faso using both methods. Agreement between the two methods and association with mosquito infection rates in membrane feeding assays were assessed. Results Intra- and inter-assay variability was larger in QT-NASBA compared to qRT-PCR, particularly at low gametocyte densities (100 gametocyte per μL). Samples collected in one of the two transmission studies had extremely low gametocyte densities by both molecular methods, which is suggestive of RNA degradation due to an unknown number of freeze–thaw cycles and illustrates the reliance of molecular gametocyte diagnostics on a reliable cold-chain. Conclusions The experiments indicate that both qRT-PCR and QT-NASBA are of value for quantifying mature gametocytes in samples collected in field studies. For both assays, estimated gametocyte densities correlated well with mosquito infection rates. QT-NASBA is less reproducible than qRT-PCR, particularly for low gametocyte densities.