Background The origins of sex differences in cardiovascular diseases are not well understood. We examined the contribution of childhood risk factors to sex differences in adult carotid artery plaques and intima‐media thickness (carotid IMT). Methods and Results Children in the 1985 Australian Schools Health and Fitness Survey were followed up when they were aged 36 to 49 years (2014–19, n=1085–1281). Log binomial and linear regression examined sex differences in adult carotid plaques (n=1089) or carotid IMT (n=1283). Childhood sociodemographic, psychosocial, and biomedical risk factors that might contribute to sex differences in carotid IMT/plaques were examined using purposeful model building with additional adjustment for equivalent adult risk factors in sensitivity analyses. Women less often had carotid plaques (10%) than men (17%). The sex difference in the prevalence of plaques (relative risk [RR] unadjusted 0.59 [95% CI, 0.43 to 0.80]) was reduced by adjustment for childhood school achievement and systolic blood pressure (RR adjusted 0.65 [95% CI, 0.47 to 0.90]). Additional adjustment for adult education and systolic blood pressure further reduced sex difference (RR adjusted 0.72 [95% CI, 0.49 to 1.06]). Women (mean±SD 0.61±0.07) had thinner carotid IMT than men (mean±SD 0.66±0.09). The sex difference in carotid IMT (β unadjusted −0.051 [95% CI, −0.061 to −0.042]) reduced with adjustment for childhood waist circumference and systolic blood pressure (β adjusted −0.047 [95% CI, −0.057 to −0.037]) and further reduced with adjustment for adult waist circumference and systolic blood pressure (β adjusted −0.034 [95% CI, −0.048 to −0.019]). Conclusions Some childhood factors contributed to adult sex differences in plaques and carotid IMT. Prevention strategies across the life course are important to reduce adult sex differences in cardiovascular diseases.