The common genetic variation of the serotonin transporter-linked polymorphic region (5-HTTLPR) has been related to depressive symptoms, in particular after stressful life events. Although it has been investigated in the past, results suggesting that the 5-HTTLPR genotype also affects hippocampal volume are often inconsistent and it remains unclear to what extent reduced hippocampal volume is influenced by the effect of stressful life events and 5-HTTLPR genotype. Moreover, sex, which is known to affect the prevalence of depression substantially, has not been taken into account when trying to disentangle the interactive effect of common genetic variation and environmental stressors on the hippocampus. We investigated this potentially relevant three-way interaction using an automatic magnetic resonance imaging (MRI)-based segmentation of the hippocampus in 357 healthy individuals. We determined the 5-HTTLPR genotype as a biallelic locus and childhood adversity (CA) using a standard questionnaire. An interaction for hippocampal volume was found between the factors sex, genotype, and severe CA (p=0.010) as well as an interaction between genotype and severe CA (p=0.007) in men only. Post hoc tests revealed that only male S'-allele carriers with severe CA had smaller hippocampi (p=0.002). Interestingly, there was no main effect of genotype in men, while female S'-allele carriers had smaller hippocampi than L'L' carriers (p=0.023). Our results indicate that sex modulates the interactive effect of the 5-HTTLPR genotype and CA on hippocampal volume. While the S'-allele is associated with hippocampal volume independent of CA in women, men only have smaller hippocampi if they carry the risk allele and experienced severe CA.