Background: The diagnosis of Coronavirus disease 2019 (COVID-19) relies on the positivity of nasopharyngeal swab. However, a significant percentage of symptomatic patients may test negative. We evaluated the reliability of COVID-19 diagnosis made by radiologists and clinicians and its accuracy versus serology in a sample of patients hospitalized for suspected COVID-19 with multiple negative swabs. Methods: Admission chest CT-scans and clinical records of swab-negative patients, treated according to the COVID-19 protocol or deceased during hospitalization, were retrospectively evaluated by two radiologists and two clinicians, respectively. Results: Of 254 patients, 169 swab-confirmed cases and one patient without chest CT-scan were excluded. A total of 84 patients were eligible for the reliability study. Of these, 21 patients died during hospitalization; the remaining 63 underwent serological testing and were eligible for the accuracy evaluation. Of the 63, 26 patients showed anti-Sars-Cov-2 antibodies, while 37 did not. The inter-rater agreement was “substantial” (kappa 0.683) between radiologists, “moderate” (kappa 0.454) between clinicians, and only “fair” (kappa 0.341) between radiologists and clinicians. Both radiologic and clinical evaluations showed good accuracy compared to serology. Conclusions: The radiologic and clinical diagnosis of COVID-19 for swab-negative patients proved to be sufficiently reliable and accurate to allow a diagnosis of COVID-19, which needs to be confirmed by serology and follow-up.