Emotional events are remembered better than neutral events possibly because the amygdala enhances the function of medial temporal lobe (MTL) memory system (modulation hypothesis). Although this hypothesis has been supported by much animal research, evidence from humans has been scarce and indirect. We investigated this issue using event-related fMRI during encoding of emotional and neutral pictures. Memory performance after scanning showed a retention advantage for emotional pictures. Successful encoding activity in the amygdala and MTL memory structures was greater and more strongly correlated for emotional than for neutral pictures. Moreover, a double dissociation was found along the longitudinal axis of the MTL memory system: activity in anterior regions predicted memory for emotional items, whereas activity in posterior regions predicted memory for neutral items. These results provide direct evidence for the modulation hypothesis in humans and reveal a functional specialization within the MTL regarding the effects of emotion on memory formation.