AbstractIntroductionMultistakeholder engagement is crucial for conducting health services research. Delphi‐based methodologies combining iterative rounds of questions with feedback on and discussion of group results are a well‐documented approach to multistakeholder engagement. This study develops hypotheses about the impact of panel composition and topic on the propensity and meaningfulness of response changes in multistakeholder modified‐Delphi panels.MethodsWe conducted three online modified‐Delphi (OMD) multistakeholder panels using the same protocol. We assigned 60 maternal and child health professionals to a homogeneous (professionals only) panel, 60 pregnant or postpartum women (patients) to a homogeneous panel, and 30 professionals and 30 patients to a mixed panel. In Round 1, participants rated the seriousness of 11 maternal and child health outcomes using a 0–100 scale and explained their ratings. In Round 2, participants saw their own and their panel's Round 1 results and discussed them using asynchronous, anonymous discussion boards moderated by the study investigators. In Round 3, participants revised their original ratings. Our outcome measures included binary indicators of response changes to ratings of the low, medium and high severity maternal and child health outcomes and their meaningfulness, measured by a change of 10 or more points.ResultsParticipants changed 818 of 1491 (55%) of responses; the majority of response changes were meaningful. Patterns of response changes were different for patients and professionals and for different levels of outcome seriousness. Using study results and the literature, we developed three hypotheses. First, OMD participants, regardless of their stakeholder group, are more likely to change their responses on preference‐sensitive topics where there is a range of viable alternatives or perspectives. Second, patients are more likely to change their responses and to do so meaningfully in mixed panels, whereas professionals are more likely to do so in homogeneous panels. Third, the association between panel composition and response change varies according to the topic (e.g., the level of outcome seriousness).ConclusionsResults of our work not only helped generate empirically derived hypotheses to be tested in future research but also offer practical recommendations for designing multistakeholder OMD panels.Patient or Public ContributionPregnant or postpartum women were involved in this study.