Abstract Purpose To describe the association between daily activity (i.e., daily step counts and accelerometer intensity measures) and serum TT levels in a representative sample of US adults aged 18 years or older. Methods A retrospective cohort study was carried out utilizing the NHANES (National Health and Nutrition Examination Survey) 2003–2004 cycle. Physical activity was measured with a waist-worn uniaxial accelerometer (AM-7164; ActiGraph) for up to 7 days using a standardized protocol. Using linear and multivariable logistic regression controlling for relevant social, demographic, lifestyle, and comorbidity characteristics, we assessed the association between daily step counts and TT. Results A total of 279 subjects with a median age 46 (IQR: 33–56) were included in the analysis. 23.3% of the cohort had a low serum TT level (TT < 350 ng/dl). Compared to men who took <4000 steps per day, men who took >4000 or >8000 steps/day had a lower odd of being hypogonadal (OR 0.14, 95% CI: 0.07–0.49 and 0.08, 95%CI: 0.02–0.44, respectively). While a threshold effect was noted on average, TT increased 7 ng/dL for each additional 1000 steps taken daily (β-estimate: 0.007, 95% CI: 0.002–0.013). Conclusions Patients with the lowest daily step counts had higher odds of being hypogonadal. The current work supports a possible association between daily steps, total testosterone, and hypogonadism for men in the US.