PurposeTobacco prevention media campaigns are an important tool to address youth tobacco use. We developed a theory-based perceived message effectiveness (PME) Scale to use when vetting messages for campaigns.MethodsParticipants were a national sample of N=623 US adolescents (ages 13–17 years) recruited from a national probability-based panel. In an online experiment, we randomised adolescents to view tobacco prevention ads. All participants viewed an ad on smoking or vaping from the US Food and Drug Administration’sThe Real Costcampaign and a control video, in a random order. After ad exposure, we assessed PME using nine candidate items and constructs for convergent and criterion validity analyses. We used confirmatory factor analysis and examined information curves to select the scale items.ResultsA brief PME scale with three items (α=0.95) worked equally well for demographically diverse adolescents with different patterns of tobacco use.The Real Costads generated higher PME scores than the control videos for both vaping and smoking (convergent validity; p<0.05). Higher PME scores were associated with greater attention, fear, cognitive elaboration and anticipated social interactions (convergent validity; r=0.31–0.66), as well as more negative attitudes toward and lower susceptibility to vaping and smoking (criterion validity; r=−0.14 to −0.37). A single-item PME measure performed similarly to the three-item version.ConclusionsThe University of North Carolina PME Scale for Youth is a reliable and valid measure of the potential effectiveness of vaping and smoking prevention ads. Employing PME scales during message development and selection may help youth tobacco prevention campaigns deploy more effective ads.