While self-reported Coronavirus Disease 2019 (COVID-19) symptom checklists have been extensively used during the pandemic, they have not been sufficiently validated from a psychometric perspective. We, therefore, used advanced psychometric modelling to explore the construct validity and internal consistency of an online self-reported COVID-19 symptom checklist and suggested adaptations where necessary. Fit to the Rasch model was examined in a sample of 1638 Austrian citizens who completed the checklist on up to 20 days during a lockdown. The items’ fatigue’, ‘headache’ and ‘sneezing’ had the highest likelihood to be affirmed. The longitudinal application of the symptom checklist increased the fit to the Rasch model. The item ‘cough’ showed a significant misfit to the fundamental measurement model and an additional dependency to ‘dry cough/no sputum production’. Several personal factors, such as gender, age group, educational status, COVID-19 test status, comorbidities, immunosuppressive medication, pregnancy and pollen allergy led to systematic differences in the patterns of how symptoms were affirmed. Raw scores’ adjustments ranged from ±0.01 to ±0.25 on the metric scales (0 to 10). Except for some basic adaptations that increases the scale’s construct validity and internal consistency, the present analysis supports the combination of items. More accurate item wordings co-created with laypersons would lead to a common understanding of what is meant by a specific symptom. Adjustments for personal factors and comorbidities would allow for better clinical interpretations of self-reported symptom data.