The severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has caused an ongoing pandemic. Reverse transcription polymerase chain reaction (RT-PCR) is the gold standard for the detection of SARS-CoV-2 and has been applied to different specimen types. Understanding the virus load and virus detection frequency in different specimen types is important to improve diagnosis and estimate the duration of potential infectivity. We conducted a retrospective single-center cohort study on hospitalized and outpatients with SARS-CoV-2 infection. We analyzed the frequency of virus detection, virus load, and duration of the virus excretion in upper and lower respiratory specimens as well as stool and plasma. We found that the frequency of SARS-CoV-2 detection, the virus load, and duration of virus excretion was higher in lower respiratory tract (LRT) than in upper respiratory tract (URT) specimens. The duration of virus excretion was longer in patients requiring intensive care unit (ICU) admission. In conclusion, LRT specimens are the most appropriate specimen type for the detection and follow-up of SARS-CoV-2 infection. Duration of virus excretion is longer in severe cases of SARS-CoV-2 infection.