Brain-derived neurotrophic factor (BDNF) is involved in memory and the pathophysiology of various neuropsychiatric disorders. A single nucleotide polymorphism in the human BDNF gene (Val66Met) affects memory, and influences Alzheimer's disease and depression vulnerability in a sex-specific manner. Recent animal studies suggest that BDNF mediates memory for emotional experiences in the amygdala, but it is currently unknown whether BDNF Val66Met influences memory processing in the amygdala. Here, we investigated its effect on the successful encoding and recognition of biologically salient stimuli. Forty-seven healthy volunteers memorized and recognized faces while their brain activity was measured with event-related functional MRI. No significant differences in memory performance were observed between Val homozygotes and Met allele carriers. The imaging results demonstrated BDNF genotype x sex interactions in the amygdala during memory formation, and in the prefrontal cortex and posterior cingulate cortex during memory retrieval. Subsequent tests showed a larger contribution of these brain regions to successful encoding and retrieval in male Met allele carriers than male Val homozygotes, whereas no significant differences were observed in females. These results provide preliminary evidence that the BDNF Val66Met polymorphism influences specific mnemonic operations underlying encoding and retrieval of salient stimuli, and suggest less efficient memory processing in male Met allele carriers. Furthermore, the sex-specific genotype effects may contribute to sex-specific effects of BDNF Val66Met on depression vulnerability.