Emotional memories are better remembered than neutral ones. The amygdala is involved in this enhancement not only by modulating the hippocampal activity, but possibly also by modulating central arousal. Using functional magnetic resonance imaging, we analyzed the retrieval of neutral faces encoded in emotional or neutral contexts. The pupillary size measured during encoding was used as a modulator of brain responses during retrieval. The interaction between emotion and memory showed significant responses in a set of areas, including the amygdala and parahippocampal gyrus. These areas responded significantly more for correctly remembered faces encoded in an emotional, compared with neutral, context. The same interaction conducted on responses modulated by the pupillary size revealed an area of the dorsal tegmentum of the ponto-mesencephalic region, consistent with the locus ceruleus. Moreover, a psychophysiological interaction showed that amygdalar responses were more tightly related to those of the locus ceruleus when remembering faces that had been encoded in an emotional, rather than neutral, context. These findings suggest that the restoration of a central arousal similar to encoding takes part in the successful retrieval of neutral events learned in an emotional context.