Background: Immunotherapy has completely changed the treatment of solid tumors. Although immune checkpoint inhibitors (ICIs) seem to be an appealing alternative to chemotherapy, especially in elderly patients, due to a more tolerable toxicity profile, they can lead to a peculiar variety of immune-related adverse events (irAEs). However, data on tolerability and outcome of ICIs in the elderly are lacking due to poor accrual in clinical trials of these patients. Methods: We performed a retro-prospective analysis on patients treated with single agent anti-PD-L1/PD-1 at the Clinical Oncology Unit, Careggi University Hospital, from March 2016 to March 2020. Data on the treatment responses, type and severity of irAEs, as well as the corticosteroids (CCS) dosage used for irAEs and the discontinuation rate, were described per each patient, according to two different age-based cohorts of patients (< or ≥70 years). Results: We reported a lower incidence of all-grade toxicity in elderly compared to younger patients (64.9% vs. 44.9%, p = 0.018). The two age-cohorts showed a different profile of irAEs. Endocrine irAEs were significantly higher in younger patients (39.7% vs. 21.7%, p = 0.002), while dermatologic toxicities were more common in the older group (35.0% vs. 11.3%, p = 0.047). Use of CCS and treatment discontinuation rate do not differ significantly between the two age groups. Conclusion: Our findings suggest that treatment with ICIs in elderly populations is safe and feasible. Patients over 70 years are more prone to develop skin irAEs, while younger patients are more subject to experience endocrine toxicities.