Background: For patients with esophageal squamous cell carcinoma (ESCC) treated with surgery alone, the incidence of local-regional recurrence remains unfavorable. Postoperative radiotherapy (PORT) has been associated with increased local-regional recurrence-free survival (LRFS), although its application is limited by concerns of PORT-related toxicities. Methods: Among 3591 patients with ESCC analyzed in this study, 2765 patients with T3-4N0 and T1-4N1-3 lesions and specific local-regional status information were analyzed in a subsequent analysis of adjuvant radiation dose (aRTD) effect. Application of the restricted cubic spline regression model revealed a non-linear relationship between aRTD and survival/radiotoxicity. Linear regression analysis (LRA) was performed to evaluate correlations between LRFS and overall survival (OS)/ disease-free survival (DFS). Results: For patients staged T1–2N0, T1–2N1–3, T3–4N0, and T3–4N1–3, 5-year OS in PORT and non-PORT groups were 77.38% vs. 72.91%, p = 0.919, 52.35% vs. 46.60%, p = 0.032, 73.41% vs. 61.19%, p = 0.005 and 38.30% vs. 25.97%, p < 0.001. With aRTD escalation, hazard ratios (HRs) of OS/DFS declined until aRTD exceeded 50Gy, then increased, whereas that of LRFS declined until aRTD exceeded 50 Gy, then remained steady. HR of treatment-related mortality was stable until aRTD exceeded 50 Gy, then increased. LRA revealed strong correlations between LRFS and OS/DFS (r = 0.984 and r = 0.952, respectively). An absolute 1% advancement in LRFS resulted in 0.32% and 0.34% improvements in OS and DFS. Conclusions: An aRTD of 50Gy was well-tolerated, with favorable survival resulting from PORT-related LRFS improvement in patients staged T3–4N0 or T1-4N1–3. Further stratification analyses based on tumor burden would help determine potential PORT-beneficiaries.