Abstract Background Many studies aggregate prescription opioid misuse (POM) and heroin use into a single phenotype, but emerging evidence suggests that their genetic and environmental influences may be partially distinct. Methods In total, 7164 individual twins (84.12% complete pairs; 59.81% female; mean age = 30.58 years) from the Australian Twin Registry reported their lifetime misuse of prescription opioids, stimulants, and sedatives, and lifetime use of heroin, cannabis, cocaine/crack, illicit stimulants, hallucinogens, inhalants, solvents, and dissociatives via telephone interview. Independent pathway models (IPMs) and common pathway models (CPMs) partitioned the variance of drug use phenotypes into general and drug-specific genetic (a), common environmental (c), and unique environmental factors (e). Results An IPM with one general a and one general e factor and a one-factor CPM provided comparable fit to the data. General factors accounted for 55% (a = 14%, e = 41%) and 79% (a = 64%, e = 15%) of the respective variation in POM and heroin use in the IPM, and 25% (a = 12%, c = 8%, e = 5%) and 80% (a = 38%, c = 27%, e = 15%) of the respective variation in POM and heroin use in the CPM. Across both models, POM emerged with substantial drug-specific genetic influence (26–39% of total phenotypic variance; 69–74% of genetic variance); heroin use did not (0% of total phenotypic variance; 0% of genetic variance in both models). Prescription sedative misuse also demonstrated significant drug-specific genetic variance. Conclusions Genetic variation in POM, but not heroin use, is predominantly drug-specific. Misuse of prescription medications that reduce experiences of subjective distress may be partially influenced by sources of genetic variation separate from illicit drug use.