Background Mental illness is a major disease burden in the world and disproportionately affects the socially disadvantaged, but studies on the longitudinal association of poverty with anxiety and stress are rare, especially in Asia. Using data from Hong Kong, we aimed to (1) assess the cross-sectional association of poverty with anxiety and stress at baseline, and (2) to examine whether baseline poverty and change in poverty status over time are associated with a subsequent change in anxiety and stress. Methods Data were obtained from two waves of a territory-wide longitudinal survey in Hong Kong, with sample sizes of n=1970 and n=1224 for baseline and follow-up, respectively. Poverty was measured with a Deprivation Index and income-poverty. Anxiety and stress symptoms were assessed using Chinese Depression, Anxiety and Stress Scale—21 Items. We conducted cross-sectional and longitudinal analyses on the association of poverty with anxiety and stress. Outcomes Deprivation, but not income-poverty, was significantly associated with both outcomes at baseline. Increased deprivation over time was associated with greater score and increased risk of anxiety and stress. Persistent deprivation over time was associated with greater anxiety and stress, and increased risk of incident anxiety. Interpretation Deprivation could have significant independent effects on anxiety and stress, even after adjusting for the effects of income-poverty. Greater attention should be paid to deprivation in policymaking to tackle the inequalities of mental health problems, especially since stress and anxiety are precursors to more severe forms of mental illness and other comorbidities.