Eosinophilic esophagitis (EoE) is an immune-mediated esophageal disorder, linked with sensitization to food and airborne allergens. Dietary manipulations are proposed for the management of EoE inflammation and are often successful, confirming the etiological role of food allergens. Three different dietary approaches are widely used: the elemental, the empirical, and the allergy-test-driven approach. We performed a systematic review to assess the evidence on the association of allergens, detected by allergy tests, with clinically confirmed triggers of EoE. We systematically searched PubMed, Scopus, Embase, and the Cochrane Library, through 1 June 2021. We sought studies examining the correlation of skin-prick tests (SPT), atopy patch tests (APT), specific IgE, and serum-specific IgG4, with confirmed triggers of EoE. Data on the use of prick–prick tests were also extracted. Evidence was independently screened by two authors against predefined eligibility criteria. Risk of bias was assessed with the ROBINS-I tool. Of 52 potentially eligible studies, 16 studies fulfilling quality criteria were included. These studies used one to three different allergy tests detecting food sensitization. The positive predictive value was generally low to moderate but higher when a combination of tests was used than single-test evaluations. None of the selected studies used serum-specific IgG4. Although an extreme methodological variability was noticed in the studies, allergy-based elimination diets were estimated to be efficient in 66.7% of the cases. The efficacy of targeted elimination diets, guided by SPT, sIgE, and/or APT allergy tests, does not appear superior to empirical ones. In the future, tests using esophageal prick testing or ex vivo food antigen stimulation may prove more efficient to guide elimination diets.