While is widely accepted that the posterior temporal region is activated during the observation of faces showing gaze shifts, it is still unclear whether its activity is stronger while observing direct or averted gaze. Furthermore, despite its assessed role in social cognition, studies describing an enhanced activity of the posterior temporal region during the observation of gaze aversion interpreted this activity in terms of spatial attention toward the target direction. This spatial attention interpretation is not easily reconcilable with the role of the posterior temporal region in social cognition, and an overarching view of its global cognitive function would be much more preferable. Here we used intracranial EEG to assess the precise spatial localization of the gaze shifts coding in the posterior temporal region, to assess its selectivity for direct versus averted gaze and to distinguish between a spatial-attentional and a social interpretations of gaze aversion. We found stronger activation during gaze aversion than direct gaze and lateral side switch observation, the latter indicating that the crucial aspect of gaze aversion is the prior presence of the eye contact and its interruption, and not the gaze direction. These results suggest a more social-oriented interpretation based on the view that among humans, gaze aversion signals a negative relational evaluation in social interaction. Hum Brain Mapp, 2013. © 2013 Wiley Periodicals, Inc.