Chronic pain and substance use disorders are serious conditions that are prevalent among homeless populations. The aim of this study was to examine the association between chronic pain and substance use among individuals experiencing homelessness and mental illness. We analyzed cross-sectional data from two sites of the At Home/Chez Soi study (Vancouver and Toronto) using bivariate statistics and multivariate logistic regression. Substance use and chronic pain parameters were assessed with the Maudsley Addiction Profile and purpose-designed short instruments. The sample comprised 828 participants. Mean age was 42.4 years and 54% reported chronic pain. In bivariate analysis, chronic pain was significantly associated with use of opioids and stimulants, daily substance use, polysubstance use and injecting as route of administration. In multivariate analysis, only daily substance use (OR: 1.46, 95% CI: 1.02–2.09) and injecting (OR: 1.81, 95% CI: 1.08–3.05) remained as significant associated factors, whereas neither use of opioids nor use of stimulants specifically were significantly associated with chronic pain. Among participants with chronic pain, daily substance users (50% vs. 22%, p < 0.001) and injectors (66% vs. 24%, p < 0.001) were more likely to use non-prescribed medication for pain. Participants with daily substance use were less likely to receive professional treatment (52% vs. 64%, p = 0.017) and prescribed pain medication (42% vs. 54%, p = 0.023). Our findings suggest an association of chronic pain with patterns related to severity of substance use rather than to specific substance use in homeless persons with mental illness. Interventions aiming at prevention and treatment of chronic pain in this population should consider severity of substance use and associated risk behavior over use of specific substances.