Abstract Mammographic features influence breast cancer risk and are used in risk prediction models. Understanding how genetics influence mammographic features is important because the mechanisms through which they are associated with breast cancer are not well known. Here, using mammographic screening history and detailed questionnaire data from 56,820 women from the KARMA prospective cohort study, we investigated the association between a genetic predisposition to breast cancer and mammographic features among women with a family history of breast cancer (N = 49,674) and a polygenic risk score (PRS, N = 9,365). The heritability of mammographic features such as dense area (MD), microcalcifications, masses, and density change (MDC, cm2/year) was estimated using 1,940 sister pairs. Heritability was estimated at 58% [95% confidence interval (CI), 48%–67%) for MD, 23% (2%–45%) for microcalcifications, and 13% (1%–25%)] for masses. The estimated heritability for MDC was essentially null (2%; 95% CI, −8% to 12%). The association between a genetic predisposition to breast cancer (using PRS) and MD and microcalcifications was positive, while for masses this was borderline significant. In addition, for MDC, having a family history of breast cancer was associated with slightly greater MD reduction. In summary, we have confirmed previous findings of heritability in MD, and also established heritability of the number of microcalcifications and masses at baseline. Because these features are associated with breast cancer risk and can improve detecting women at short-term risk of breast cancer, further investigation of common loci associated with mammographic features is warranted to better understand the etiology of breast cancer. Significance: These findings provide novel data on the heritability of microcalcifications, masses, and density change, which are all associated with breast cancer risk and can indicate women at short-term risk.