Background Physical activity is associated with reduced cardiovascular disease risk, mainly through effects on atherosclerosis. Aortic stiffness may be an alternative mechanism. We examined whether patterns of physical activity and sedentary behavior are associated with rate of aortic stiffening. Methods and Results Carotid–femoral pulse wave velocity ( PWV ) was measured twice using applanation tonometry at mean ages 65 (in 2008/2009) and 70 (in 2012/2013) years in the Whitehall‐ II study (N=5196). Physical activity was self‐reported at PWV baseline (2008/2009) and twice before (in 1997/1999 and 2002/2003). Sedentary time was defined as sitting time watching television or at work/commute. Linear mixed models adjusted for metabolic and lifestyle risk factors were used to analyze PWV change. Mean ( SD ) PWV (m/s) was 8.4 (2.4) at baseline and 9.2 (2.7) at follow‐up, representing a 5‐year increase of 0.76 m/s (95% CI 0.69, 0.83). A smaller 5‐year increase in PWV was observed for each additional hour/week spent in sports activity (−0.02 m/s [95% CI −0.03, −0.001]) or cycling (−0.02 m/s [−0.03, −0.008]). Walking, housework, gardening, or do‐it‐yourself activities were not significantly associated with aortic stiffening. Each additional hour/week spent sitting was associated with faster PWV progression in models adjusted for physical activity (0.007 m/s [95% CI 0.001, 0.013]). Increasing physical activity over time was associated with a smaller subsequent increase in PWV (−0.16 m/s [−0.32, −0.002]) compared with not changing activity levels. Conclusions Higher levels of moderate‐to‐vigorous physical activity and avoidance of sedentary behavior were each associated with a slower age‐related progression of aortic stiffness independent of conventional vascular risk factors.