We evaluated the accuracy of coronary artery calcium (CAC) scoring on a dual-source photon-counting detector CT (PCD-CT). An anthropomorphic chest phantom underwent ECG-gated sequential scanning on a PCD-CT at 120 kV with four radiation dose levels (CTDIvol, 2.0–8.6 mGy). Polychromatic images at 120 kV (T3D) and virtual monoenergetic images (VMI), from 60 to 75 keV without quantum iterative reconstruction (no QIR) and QIR strength levels 1–4, were reconstructed. For reference, the same phantom was scanned on a conventional energy-integrating detector CT (120 kV; filtered back projection) at identical radiation doses. CAC scoring in 20 patients with PCD-CT (120 kV; no QIR and QIR 1–4) were included. In the phantom, there were no differences between CAC scores of different radiation doses (all, p > 0.05). Images with 70 keV, no QIR (CAC score, 649); 65 keV, QIR 3 (656); 65 keV; QIR4 (648) and T3D, QIR4 (656) showed a <1% deviation to the reference (653). CAC scores significantly decreased at increasing QIR levels (all, p < 0.001) and for each 5 keV-increase (all, p < 0.001). Patient data (median CAC score: 86 [inter-quartile range: 38–978] at 70 keV) confirmed relationships and differences between reconstructions from the phantom. First phantom and in-vivo experience with a clinical dual-source PCD-CT system shows accurate CAC scoring with VMI reconstructions at different radiation dose levels.