Background Regular exercise training represents an important modifier of arterial stiffness (AS). Therefore, sex‐specific relations between domains of physical activity (PA; commuting, domestic, and leisure‐time PA, including active sport and occupational PA) with AS were investigated. Methods and Results Stiffness index by digital photoplethysmography was investigated in 12 650 subjects from the GHS (Gutenberg Health Study). Self‐reported PA was evaluated by the “Short Questionnaire to Assess Health‐Enhancing Physical Activity” and reported as activity score peer week, being a combined measure of duration, frequency, and intensity of PA. Multivariable linear regression analysis demonstrated strong beneficial effects of repetitive activities, such as active commuting or leisure‐time PA–related walking on AS in men, but not in women. Lower AS associated with endurance training was also found among men and premenopausal women. In contrast, intense occupational PA was related to stiffer vessels in men ( P <0.0001) and women ( P =0.0021) in a fully adjusted model. Combination of both, performing endurance training and having stiffness index values below median, resulted in the best survival. In contrast, subjects with elevated stiffness index at baseline without any endurance activities demonstrated the worst survival. Conclusions In this population representative sample, a differential impact of domains of self‐reported PA on AS was demonstrated. Our data strengthen the importance of regular endurance PA to induce a reduction of AS, which, in turn, may improve cardiovascular prognosis. We also report deleterious effects of intense occupational PA on stiffness index, a finding that needs further confirmation by larger prospective trials.