The brown marmorated stink bug, Halyomorpha halys (Stål), is an invasive species causing economic crop losses. This species was recently detected attacking olive fruits. The aim of this study was to characterize feeding damage. Olive samples were initially collected from a field where H. halys was reported to cause damage to olive fruits. Hence, we conducted a field trial on the Moraiolo variety using sleeve cages to test the effect of H. halys feeding pressure on olive fruit drop and evaluated the effect of feeding on fruit quality. We tested two densities of H. halys (two or eight adults/cage) at two different stages of olive development, pre- and post-pit hardening. High pressure of H. halys before pit hardening caused a significant fruit drop compared to the control. In addition, chemical analysis of damaged and infested fruits revealed higher levels of total phenols compared to healthy fruits. These findings indicate that feeding by H. halys induced a stress response in the plants that could translate in quality variations in the olive drupes.