Abstract Background Grade group (GG) 4 prostate cancer (PC) is considered a single entity; however, there are questions regarding prognostic heterogeneity. This study assessed the prognostic differences among various Gleason scores (GSs) classified as GG 4 PC on biopsy before radical prostatectomy (RP). Methods We conducted a multicenter retrospective study, and a total of 1791 patients (GS 3 + 5: 190; GS 4 + 4: 1557; and GS 5 + 3: 44) with biopsy GG 4 were included for analysis. Biochemical recurrence (BCR)-free survival, cancer-specific survival, and overall survival were analyzed using the Kaplan–Meier method and the log-rank test. Logistic regression analysis was performed to identify factors associated with high-risk surgical pathologic features. Cox regression models were used to analyze time-dependent oncologic endpoints. Results Over a median follow-up of 75 months, 750 patients (41.9%) experienced BCR, 146 (8.2%) died of any causes, and 57 (3.2%) died of PC. Biopsy GS 5 + 3 was associated with significantly higher rates of GS upgrading in RP specimens than GS 3 + 5 and GS 4 + 4. On multivariable analysis adjusted for clinicopathologic features, different GSs within GG 4 were significantly associated with BCR (p = 0.03) but not PC-specific or all-cause mortality. Study limitations include the lack of central pathological specimen evaluation. Conclusions Patients with GG 4 at biopsy exhibited some limited biological and clinical heterogeneity. Specifically, GS 5 + 3 had an increased risk of GS upgrading. This can help individualize patients’ counseling and encourage further study to refine biopsy specimen-based GG classification.