The high infection rate and rapid spread of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) make it a world-wide pandemic. Individuals infected by the virus exhibited different degrees of symptoms, and most convalescent individuals have been shown to develop both cellular and humoral immune responses. However, virus-specific adaptive immune responses in severe patients during acute phase have not been thoroughly studied. Here, we found that in a group of COVID-19 patients with acute respiratory distress syndrome (ARDS) during hospitalization, most of them mounted SARS-CoV-2-specific antibody responses, including neutralizing antibodies. However, compared to healthy controls, the percentages and absolute numbers of both NK cells and CD8⁺ T cells were significantly reduced, with decreased IFNγ expression in CD4⁺ T cells in peripheral blood from severe patients. Most notably, their peripheral blood lymphocytes failed in producing IFNγ against viral proteins. Thus, severe COVID-19 patients at acute infection stage developed SARS-CoV-2-specific antibody responses but were impaired in cellular immunity, which emphasizes on the role of cellular immunity in COVID-19.