Aging and overweight increase the risk of developing type 2 diabetes mellitus. In this cross-sectional study, we aimed to investigate the potential mediating role of T-EMRA cells and inflammatory markers in the development of a decreased insulin sensitivity. A total of 134 healthy older volunteers were recruited (age 59.2 (SD 5.6) years). T cell subpopulations were analyzed by flow cytometry. Furthermore, body composition, HOMA-IR, plasma tryptophan (Trp) metabolites, as well as cytokines and adipokines were determined. Using subgroup and covariance analyses, the influence of BMI on the parameters was evaluated. Moreover, correlation, multiple regression, and mediation analyses were performed. In the subgroup of participants with obesity, an increased proportion of CD8+EMRA cells and elevated concentrations of plasma kynurenine (KYN) were found compared to the lower-weight subgroups. Linear regression analysis revealed that an elevated HOMA-IR could be predicted by a higher proportion of CD8+EMRA cells and KYN levels. A mediation analysis showed a robust indirect effect of the Waist-to-hip ratio on HOMA-IR mediated by CD8+EMRA cells. Thus, the deleterious effects of abdominal obesity on glucose metabolism might be mediated by CD8+EMRA cells in the elderly. Longitudinal studies should validate this assumption and analyze the suitability of CD8+EMRA cells as early predictors of incipient prediabetes.