Background— Coronary artery calcification (CAC) is widely regarded as a cumulative lifetime measure of atherosclerosis, but it remains unclear what is the relationship between calcification and traditional risk factors for coronary artery disease (CAD) and myocardial infarction (MI). This study characterizes the genetic architecture of CAC by evaluating the overall impact of common alleles associated with CAD/MI and its traditional risk factors. Methods and Results— On the basis of summary-association results from the CARDIoGRAMplusC4D study of CAD/MI, we calculated polygenic risk scores in 2599 participants of the Dutch and Belgian Lung Cancer Screening (NELSON) trial, in whom quantitative CAC levels (Agatston scores) were determined from chest computerized tomographic imaging data. The most significant polygenic model explained ≈14% of the observed CAC variance ( P =1.6×10 ^–11 ), which points to a residual effect because of many as yet unknown loci that overlap between CAD/MI and CAC. In addition, we constructed risk scores based on published single-nucleotide polymorphism associations for traditional cardiovascular risk factors and tested these scores for association with CAC. We found nominally significant associations for genetic risk scores of low-density lipoprotein-cholesterol, total cholesterol, and body mass index, which were successfully replicated in 2182 individuals of the Heinz Nixdorf Recall Study. Conclusions— Pervasive polygenic sharing between CAC and CAD/MI suggests that a substantial fraction of the heritable risk for CAD/MI is mediated through arterial calcification. We also provide evidence that genetic variants associated with serum lipid levels and body mass index influence CAC levels.