The role of empiric antifungals for post-surgical abscesses (PSAs) is controversial, and international guidelines on invasive mycoses focus on bloodstream infections. We analyzed a retrospective cohort of 319 patients with PSA at a tertiary-level hospital in Italy during the years 2013–2018. Factors associated with empiric antifungal administration were analyzed and compared with factors associated with fungal isolation from the abdomen. Forty-six patients (14.4%) received empiric antifungals (65.2% azoles). Candida was isolated in 34/319 (10.7%) cases, always with bacteria. Only 11/46 patients receiving empirical antifungals had abdominal Candida. Only 11/34 patients with a fungal isolate received empiric antifungal therapy. Upper GI surgery (OR: 4.76 (CI: 1.95–11.65), p = 0.001), an intensive care unit stay in the previous 90 days (OR: 5.01 (CI: 1.63–15.33), p = 0.005), and reintervention within 30 days (OR: 2.52 (CI: 1.24–5.13), p = 0.011) were associated with empiric antifungals in a multivariate analysis, while pancreas/biliary tract surgery was associated with fungal isolation (OR: 2.25 (CI: 1.03–4.91), p = 0.042), and lower GI surgery was protective (OR: 0.30 (CI: 0.10–0.89), p = 0.029) in a univariate analysis. The criteria for empiric antifungal therapy in our practice seem to be inconsistent with the risk factors for actual fungal isolation. Better guidance for empiric therapy should be provided by wider studies.