Sedentarism is a risk factor for cardiovascular disease (CVD), but currently it is not clear how a sedentary behavior such as long sitting time can affect atherosclerosis development. This study examined the relationship between sitting time and the prevalence of carotid and femoral subclinical atherosclerosis. A cross-sectional analysis based on a subsample of 2082 participants belonging to the Aragon Workers’ Health Study was carried out. Ultrasonography was used to assess the presence of plaques in carotid and femoral territories; the validated Spanish version of the questionnaire on the frequency of engaging in physical activity used in the Nurses’ Health Study and the Health Professionals’ was used to assess physical activity and sitting time; and demographic, anthropometric, and clinical data were obtained by trained personnel during the annual medical examination. Participants were categorized into <9 h/day and ≥9 h/day sitting time groups. After adjusting for several confounders, compared with participants that remain seated <9 h/day, those participants who remain seated ≥9 h/day had, respectively, OR = 1.25 (95%CI: 1.01, 1.55, p < 0.05) and OR = 1.38 (95%CI: 1.09, 1.74, p < 0.05) for carotid and any-territory plaque presence. Remaining seated ≥9 h/day is associated with higher odds for carotid and any-territory plaque presence independently of physical activity levels and other cardiovascular risk factors.