Immune checkpoint inhibitors (ICIs) have improved overall survival for cancer patients, however, optimal duration of ICI therapy has yet to be defined. Given ICIs were first used to treat patients with metastatic melanoma, a condition that at the time was incurable, little attention was initially paid to how much therapy would be needed for a durable response. As the early immunotherapy trials have matured past 10 years, a significant per cent of patients have demonstrated durable responses; it is now time to determine whether patients have been overtreated, and if durable remissions can still be achieved with less therapy, limiting the physical and financial toxicity associated with years of treatment. Well-designed trials are needed to identify optimal duration of therapy, and to define biomarkers to predict who would benefit from shorter courses of immunotherapy. Here, we outline key questions related to health, financial and societal toxicities of over treating with ICI and present four unique clinical trials aimed at exposing criteria for early cessation of ICI. Taken together, there is a serious liability to overtreating patients with ICI and future work is warranted to determine when it is safe to stop ICI.