Adult granulosa cell tumor (AGCT) is a rare ovarian cancer subtype, with a peak incidence around 50–55 years. Although AGCT can occur in specific syndromes, a genetic predisposition for AGCT has not been identified. The aim of this study is to identify a genetic variant in families with AGCT patients, potentially contributing to tumor evolution. We identified four families, each including two women diagnosed with AGCT. Whole-genome sequencing was performed to identify overlapping germline variants or affected genes. Familial relationship was evaluated using genealogy and genomic analyses. Patient characteristics, medical (family) history, and pedigrees were collected. Findings were compared to a reference group of 33 unrelated AGCT patients. Mean age at diagnosis was 38 years (range from 17 to 60) versus 51 years in the reference group, and seven of eight patients were premenopausal. In two families, three first degree relatives were diagnosed with breast cancer. Furthermore, polycystic ovary syndrome (PCOS) and subfertility was reported in three families. Predicted deleterious variants in PIK3C2G, BMP5, and LRP2 were identified. In conclusion, AGCTs occur in families and could potentially be hereditary. In these families, the age of AGCT diagnosis is lower and cases of breast cancer, PCOS, and subfertility are present. We could not identify an overlapping genetic variant or affected locus that may explain a genetic predisposition for AGCT.