Abstract The recent report of the Intergovernmental Panel on Climate Change is stark in its warnings about the changing climate, including future increases in the frequency and intensity of extremely hot weather. The well-established impacts of extreme heat on human health have led to widespread implementation of national and city-wide heat plans for mitigating such impacts. Evaluations of the effectiveness of some heat plans have been published, with previous reviews highlighting key methodological challenges. This article reviews methods used since and that address those challenges, so helping to set an agenda for improving evaluations of heat plans in terms of their effectiveness in reducing heat-health impacts. We examined the reviews that identified the methodological challenges and systematically searched the literature to find evaluations that had since been conducted. We found 11 evaluations. Their methods help address the key challenge of identifying study control groups and address other challenges to a limited extent. For future evaluations, we recommend: utilising recent evaluation methodologies, such as difference-in-differences quasi-experimental designs where appropriate; cross-agency working to utilise data on morbidity and confounders; adoption of a proposed universal heat index; and greater publication of evaluations. More evaluations should assess morbidity outcomes and be conducted in low- and middle-income countries. Evaluations of heat plans globally should employ robust methodologies, as demonstrated in existing studies and potentially transferrable from other fields. Publication of such evaluations will advance the field and thus help address some of the health challenges resulting from our changing climate.