Immune checkpoint inhibitors have dramatically revolutionized the therapeutic landscape for patients with advanced malignancies. Recently, convincing evidence has shown meaningful influence of gut microbiome on human immune system. With the complex link between gut microbiome, host immunity and cancer, the variations in the gut microbiota may influence the efficacy of immune checkpoint inhibitors. Indeed, some bacterial species have been reported to be predictive for cancer outcome in patients treated with immune checkpoint inhibitors. Although immune checkpoint inhibitors are currently proven to be an effective anti-tumor treatment, they can induce a distinct form of toxicity, termed immune-related adverse events. Immune-related colitis is one of the common toxicities from immune checkpoint inhibitors, and it might preclude the cancer therapy in severe or refractory cases. The manipulation of gut microbiome by fecal microbiota transplantation or probiotics administration has been suggested as one of the methods to enhance anti-tumor effects and decrease the risk of immune-related colitis. Here we review the role of gut microbiome on immune checkpoint inhibitor therapy and consequent immune-related colitis to provide a new insight for better anti-cancer therapy.