Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection results in rapid T lymphocytopenia and functional impairment of T cells. The underlying mechanism, however, remains incompletely understood. In this study, we focused on characterizing the phenotype and kinetics of T-cell subsets with mitochondrial dysfunction (MD) by multicolor flow cytometry and investigating the association between MD and T-cell functionality. While 73.9% of study subjects displayed clinical lymphocytopenia upon hospital admission, a significant reduction of CD4 or CD8 T-cell frequency was found in all asymptomatic, symptomatic, and convalescent cases. CD4 and CD8 T cells with increased MD were found in both asymptomatic and symptomatic patients within the first week of symptom onset. Lower proportion of memory CD8 T cell with MD was found in severe patients than in mild ones at the stage of disease progression. Critically, the frequency of T cells with MD in symptomatic patients was preferentially associated with CD4 T-cell loss and CD8 T-cell hyperactivation, respectively. Patients bearing effector memory CD4 and CD8 T cells with the phenotype of high MD exhibited poorer T-cell responses upon either phorbol 12-myristate-13-acetate (PMA)/ionomycin or SARS-CoV-2 peptide stimulation than those with low MD. Our findings demonstrated an MD-associated mechanism underlying SARS-CoV-2-induced T lymphocytopenia and functional impairment during the acute phase of infection.