Human faces capture attention, provide information about group belonging, and elicit automatic prepared responses. Early experiences with other-race faces play a critical role in acquiring face expertise, but the exact mechanism through which early experience exerts its influence is still to be elucidated. Genetic factors and a multi-ethnic context are likely involved, but their specific influences have not been explored. This study investigated how oxytocin receptor gene (OXTR) genotypes and childcare experience interacted to regulate face categorization in adults. Information about single nucleotide polymorphisms of OXTR (rs53576) and experiences with own- and other-race child caregivers was collected from 89 Singaporean adults, who completed a visual categorization task with own- versus other-race faces. Participants were grouped into A/A homozygotes and G carriers and assigned a score to account for their type of child caregiver experience. A multivariate linear regression model was used to estimate the effect of genetic group, child caregiver experience, and their interaction on categorization reaction time. A significant interaction of genetic group and child caregiver experience (t = 2.48, p = 0.015), as well as main effects of both genetic group (t = −2.17, p = 0.033) and child caregiver experience (t = −4.29, p < 0.001) emerged. Post-hoc analysis revealed that the correlation between categorization reaction time and child caregiver experience was significantly different between the two genetic groups. A significant gene x environment interaction on face categorization appears to represent an indirect pathway through which genes and experiences interact to shape mature social sensitivity to faces in human adults.