Female genital schistosomiasis (FGS) can occur in S. haematobium infection and is caused by parasite egg deposition in the genital tract. Confirming a diagnosis of FGS is challenging due to the lack of a diagnostic reference standard. A 2010 expert-led consensus meeting proposed visual inspection of the cervicovaginal mucosa as an adequate reference standard for FGS diagnosis. The agreement of expert human reviewers for visual-FGS has not been previously described. Methods: In two Zambian communities, non-menstruating, non-pregnant, sexually-active women aged 18-31 years participating in the HPTN 071 (PopART) Population-Cohort were enrolled in a cross-sectional study. Self-collected genital swabs and a urine specimen were collected at a home visit; trained midwives performed CVL and hand-held colposcopy at a clinic visit. S. haematobium eggs and circulating anodic antigen (CAA) were detected from urine. Two expert reviewers independently diagnosed visual-FGS as the presence of sandy patches, rubbery papules or abnormal blood vessels in digital cervicovaginal images obtained by hand-held colposcopy. PCR-FGS was defined as Schistosoma DNA detected by real-time PCR in any genital specimen (CVL or genital swab). Results: Of 527 women with cervicovaginal colposcopic images, 468/527 (88.8%) were deemed interpretable by Reviewer 1 and 417/527 (79.1%) by Reviewer 2. Visual-FGS was detected in 35.3% (165/468) of participants by expert review of colposcopic images by Reviewer 1 and in 63.6% (265/417) by Reviewer 2. Cohen’s kappa statistic for agreement between the two expert reviewers was 0.16, corresponding to "slight" agreement. The reviewers made concordant diagnoses in 38.7% (204/527) participants (100 negative, 104 positive) and discordant diagnoses in 31.8% (168/527) participants. Conclusions: The unexpectedly low level of correlation between expert reviewers highlights the imperfect nature of visual diagnosis for FGS based on cervicovaginal images obtained with a hand-held colposcope. This finding is a call to action for improved point-of-care diagnostics for female genital schistosomiasis