Rapid antigen tests (RATs) are an integral part of SARS-CoV-2 containment strategies. As emerging variants of concern (VOCs) displace the initially circulating strains, it is crucial that RATs do not fail to detect these new variants. In this study, four RATs for nasal swab testing were investigated using cultured strains of B.1.1 (non-VOC), B.1.1.7 (Alpha), B.1.351 (Beta), P.1 (Gamma), and B.1.617.2 (Delta). Based on dilution series in cell culture medium and pooled saliva, the limit of detection of these RATs was determined in a laboratory setting. Further investigations on cross-reactivity were conducted using recombinant N-protein from seasonal human coronaviruses (hCoVs). RATs evaluated showed an overall comparable performance with cultured strains of the non-VOC B.1.1 and the VOCs Alpha, Beta, Gamma, and Delta. No cross-reactivity was detected with recombinant N-protein of the hCoV strains HKU1, OC43, NL63, and 229E. A continuous evaluation of SARS-CoV-2 RAT performance is required, especially with regard to evolving mutations. Moreover, cross-reactivity and interference with pathogens and other substances on the test performance of RATs should be consistently investigated to ensure suitability in the context of SARS-CoV-2 containment.