Thoracic aortic calcium (TAC) appears to be a subclinical marker of cardiovascular disease (CVD) and to predict cardiovascular (CV) mortality. However, studies on TAC use tomographic scans obtained for coronary artery calcium (CAC) score, which does not include the aortic arch. This study evaluates TAC prevalence in aortic arch (AAC), ascending (ATAC) and descending thoracic aorta (DTAC) and verify whether they are associated with the same CV risk factors. Cross-sectional analysis, including 2427 participants (mean age 55.6 ± 8.7; 54.1% women) of the ELSA-Brasil cohort. Nonenhanced ECG-gated tomographies were performed in 2015–2016. Multivariable logistic regression estimated the CV risk factors associated with calcium in each segment. Overall prevalence of ATAC, AAC and DTAC was, 23.1%, 62.1%, and 31.2%, respectively. About 90.4% of the individuals with TAC had AAC and only 19.5% had calcium in all segments. In the multivariable analysis, increasing age, lower levels of schooling, current smoking, higher body mass index, and hypertension remained associated with calcium in all segments. No sex or race/ethnicity differences were found in any aortic segment. Diabetes and dyslipidemia were associated with ATAC and DTAC, but not with AAC, suggesting that AAC may reflect an overlap of mechanisms that impact vascular health, including atherosclerosis.