Purpose: Inter-tumoral heterogeneity at the differential lesion level raises the possibility of distinct organ-specific responses to immune checkpoint inhibitors (ICIs). We aimed to comprehensively examine the clinicopathological factors to predict and assess the efficacy of nivolumab, programmed cell death protein 1 (PD-1) blockade at an individual tumor site-specific level in patients with advanced hepatocellular carcinoma (aHCC). Patients and Methods: We enrolled 261 aHCC patients treated with nivolumab between 2012 and 2018. Eighty-one clinicopathological factors were comprehensively collected and analyzed. The association between all variables and survival outcomes was evaluated. According to tumor site, the organ-specific responses were assessed based on the Response Evaluation Criteria in Solid Tumors, version 1.1. Results: The liver was the most commonly involved organ (75.1%), followed by the lungs (37.5%) and lymph nodes (LNs, 11.5%). The liver of nonresponders was more frequently the organ of progression, while the lungs of responders were more frequently the organs of response. Among the 455 individual lesions (liver, n = 248; lung, n = 124; LN, n = 35; others including bone or soft tissues, n = 48), intrahepatic tumors showed the least response (10.1%), followed by lung (24.2%) and LN tumors (37.1%), indicating the presence of distinct organ-specific responses to nivolumab. In intrahepatic tumors, the organ-specific response rate decreased as the size increased (13% for ⩽50 mm, 8.1% for 50–100 mm, and 5.5% for >100 mm). In the subgroup analysis according to tumor location, patients with lung only metastasis (⩾30 mm) showed the best progression-free survival (PFS) and overall survival (OS). In contrast, primary HCC (⩾100 mm) without lung metastasis had the worst PFS and OS. Comprehensive analyses also revealed that liver function and systemic inflammatory indices, such as neutrophil-to-lymphocyte ratio (NLR), were significantly associated with PFS and OS. Conclusion: The presence and size of liver tumors, liver function, and NLR are key factors determining the response to nivolumab in aHCC. These clinical factors should be considered when treating patients with advanced HCC with PD-1 blockade.