Psychological disturbances are frequent following COVID-19. However, there is not much information about whether pre-existing psychological disorders are associated with the severity and evolution of COVID-19. We aimed to explore the associations between regular psychotropic medication use (PM) before infection as a proxy for mood or anxiety disorders with COVID-19 recovery trajectories. We used data from the Predi-COVID study. We followed adults, tested positive for SARS-CoV-2 and collected demographics, clinical characteristics, comorbidities and daily symptoms 14 days after inclusion. We calculated a score based on 16 symptoms and modeled latent class trajectories. We performed polynomial logistic regression with PM as primary exposure and the different trajectories as outcome. We included 791 participants, 51% were men, and 5.3% reported regular PM before infection. We identified four trajectories characterizing recovery dynamics: “Almost asymptomatic,” “Quick recovery,” “Slow recovery,” and “Persisting symptoms“. With a fully adjusted model for age, sex, socioeconomic, lifestyle and comorbidity, we observed associations between PM with the risks of being in more severe trajectories than “Almost Asymptomatic”: “Quick recovery” (relative risk (95% confidence intervals) 3.1 (2.7, 3.4), “Slow recovery” 5.2 (3.0, 9.2), and “Persisting symptoms“11.7 (6.9, 19.6) trajectories. We observed a gradient of risk between PM before the infection and the risk of slow or no recovery in the first 14 days. These results suggest that a pre-existing psychological condition increases the risk of a poorer evolution of COVID-19 and may increase the risk of Long COVID. Our findings can help to personalize the care of people with COVID-19.