No real-world, long-term outcomes of immunotherapy with nivolumab for recurrent or metastatic squamous cell carcinoma of the head and neck (R/M SCCHN) have yet been reported. Furthermore, the prognostic impact of the best overall response (BOR) of this therapy remains unclear. We conducted a multi-institutional cohort study of the long-term efficacy and safety of this therapy and investigated prognostic factors associated with survival. Further, we evaluated the relationship between BOR and survival. Median follow-up time was 25.9 months. Median overall survival (OS) was 9.6 months, and two-year survival rate was 25.0%. Median progression-free survival (PFS) was 3.7 months, and two-year PFS rate was 19.6%. BOR was assessed as complete response (CR) in 6%, partial response (PR) in 13%, stable disease (SD) in 30%, and progressive disease (PD) in 52% of the patients. Overall response rate was 18%, and disease control rate was 48%. For immune-related adverse events (irAEs), 38 irAEs were detected in 29 patients. On multivariate analysis, the development of irAEs was significantly associated with better OS and PFS. Better BOR was significantly associated with longer OS and PFS. These findings demonstrate the long-term efficacy and safety of nivolumab therapy for R/M SCCHN in a real-world setting. The magnitude of BOR and the development of irAEs might be useful surrogate markers of survival.