Restricting animal-based products from diet may exert beneficial effects on weight status; however, less is known about such a diet and emotional health. Moreover, personality traits, for example high neuroticism, may contribute to restrictive eating habits and potentially confound diet-health associations. We aim to systematically assess if restrictive dietary intake of animal-based products relates to lower weight and higher depressive symptoms, and if differences in personality traits play a significant role. Cross-sectional data from the baseline LIFE-Adult study were collected from 2011–2014 in Leipzig, Germany (n = 8943). Main outcomes of interest were dietary frequency of animal-derived products in the last year measured using a Food Frequency Questionnaire (FFQ), body-mass-index (BMI) (kg/m2), and the Center of Epidemiological Studies Depression Scale (CES-D). Personality traits were assessed in a subsample of n = 7906 using the Five Factor Inventory (NEO-FFI). Higher restriction of animal-based product intake was associated with a lower BMI, but not with depression scores. Personality, i.e., lower extraversion, was related to lower frequency of animal product intake. Moreover, personality traits were significantly associated with depressive symptoms, i.e., higher neuroticism, lower extraversion, lower agreeableness, lower conscientiousness, and with higher BMI. These findings encourage future longitudinal studies to test the efficacy of restricting animal-based products as a preventive and therapeutic strategy for overweight and obesity.