The coronavirus disease 2019 (COVID-19) pandemic, caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), continues to spread globally. Although several rapid commercial serological assays have been developed, little is known about their performance and accuracy in detecting SARS-CoV-2-specific antibodies in COVID-19 patient samples. Here, we have evaluated the performance of seven commercially available rapid lateral flow immunoassays (LFIA) obtained from different manufacturers, and compared them to in-house developed and validated ELISA assays for the detection of SARS-CoV-2-specific IgM and IgG antibodies in RT-PCR-confirmed COVID-19 patients. While all evaluated LFIA assays showed high specificity, our data showed a significant variation in sensitivity of these assays, which ranged from 0% to 54% for samples collected early during infection (3–7 days post symptoms onset) and from 54% to 88% for samples collected at later time points during infection (8–27 days post symptoms onset). Therefore, we recommend prior evaluation and validation of these assays before being routinely used to detect IgM and IgG in COVID-19 patients. Moreover, our findings suggest the use of LFIA assays in combination with other standard methods, and not as an alternative.