Insect bite hypersensitivity (IBH), which is a cutaneous allergic reaction to antigens from Culicoides spp., is the most prevalent skin disorder in horses. Misdiagnosis is possible, as IBH is usually diagnosed based on clinical signs. Our study is the first to employ IgE levels against several recombinant Culicoides spp. allergens as an objective, independent, and quantitative phenotype to improve the power to detect genetic variants that underlie IBH. Genotypes of 200 Shetland ponies, 127 Icelandic horses, and 223 Belgian Warmblood horses were analyzed while using a mixed model approach. No single-nucleotide polymorphism (SNP) passed the Bonferroni corrected significance threshold, but several regions were identified within and across breeds, which confirmed previously identified regions of interest and, in addition, identifying new regions of interest. Allergen-specific IgE levels are a continuous and objective phenotype that allow for more powerful analyses when compared to a case-control set-up, as more significant associations were obtained. However, the use of a higher density array seems necessary to fully employ the use of IgE levels as a phenotype. While these results still require validation in a large independent dataset, the use of allergen-specific IgE levels showed value as an objective and continuous phenotype that can deepen our understanding of the biology underlying IBH.