Objective The aim of the study is to explore possible associations according to quality of sleep and work from home (WFH) with physical inactivity (PI) during the COVID-19 pandemic. Methods This is a cross-sectional study in two Brazilian cities (>18 years), by probability sampling. Leisure-time physical activity was self-reported and classified according to guidelines. Sleep was by applying the Pittsburgh Sleep Quality Index. Work from home was classified as: total WFH or no WFH. Results A total of 1750 adults were analyzed, 69.1% were physically inactive, 51.9% had poor sleep quality, and 79.8% were not in WFH. PI was associated with poor sleep quality and non-WFH. Compared with those who reported WFH and good sleep quality, those who reported non-WFH and poor sleep quality had 4 times the prevalence of PI. Conclusions Poor sleep quality, and not WFH, is associated with leisure PI. These factors combined increase the probability of PI.