This cross-sectional multilevel study aims at investigating the associations between psychosocial working conditions of different workplace levels and perceived physical exertion among eldercare workers. Data were obtained from the ‘Danish Observational Study of Eldercare work and musculoskeletal disorderS’ (DOSES) study, including 536 eldercare workers, nested in 126 wards and 20 nursing homes. Psychosocial working conditions were measured by the Copenhagen Psychosocial Questionnaire (COPSOQ). The physical workload was measured with a self-administered scale (0–10) rating perceived physical exertion. Multilevel linear mixed models were used to determine associations of psychosocial conditions between nursing homes, wards, and workers with physical exertion. Most of the variance in the perceived physical exertion was explained by differences between workers (83%), but some variance was explained by wards (11%) and nursing homes (6%). Workers employed in nursing homes with low influence (p = 0.01) and poor leadership (p = 0.02), and in wards with high quantitative demands (p = 0.03), high work pace (p < 0.001), and low justice (p = 0.01) were at increased risk of reporting higher physical exertion. The strongest associations were found for low influence, low quality of leadership, and high work pace at nursing homes and ward levels. In conclusion, improving specific psychosocial working conditions at nursing home and ward levels may be of particular importance to reduce excessive physical workload in eldercare workers.