Abstract Background The coronavirus disease 2019 (COVID-19) pandemic might affect mental health. Data from population-representative panel surveys with multiple waves including pre-COVID data investigating risk and protective factors are still rare. Methods In a stratified random sample of the German household population (n = 6684), we conducted survey-weighted multiple linear regressions to determine the association of various psychological risk and protective factors assessed between 2015 and 2020 with changes in psychological distress [(PD; measured via Patient Health Questionnaire for Depression and Anxiety (PHQ-4)] from pre-pandemic (average of 2016 and 2019) to peri-pandemic (both 2020 and 2021) time points. Control analyses on PD change between two pre-pandemic time points (2016 and 2019) were conducted. Regularized regressions were computed to inform on which factors were statistically most influential in the multicollinear setting. Results PHQ-4 scores in 2020 (M = 2.45) and 2021 (M = 2.21) were elevated compared to 2019 (M = 1.79). Several risk factors (catastrophizing, neuroticism, and asking for instrumental support) and protective factors (perceived stress recovery, positive reappraisal, and optimism) were identified for the peri-pandemic outcomes. Control analyses revealed that in pre-pandemic times, neuroticism and optimism were predominantly related to PD changes. Regularized regression mostly confirmed the results and highlighted perceived stress recovery as most consistent influential protective factor across peri-pandemic outcomes. Conclusions We identified several psychological risk and protective factors related to PD outcomes during the COVID-19 pandemic. A comparison of pre-pandemic data stresses the relevance of longitudinal assessments to potentially reconcile contradictory findings. Implications and suggestions for targeted prevention and intervention programs during highly stressful times such as pandemics are discussed.