Abstract Background Mounting evidence indicates that humans have significant difficulties in understanding emotional expressions from individuals of different ethnic backgrounds, leading to reduced recognition accuracy and stronger amygdala activation. However, the impact of gender on the behavioral and neural reactions during the initial phase of cultural assimilation has not been addressed. Therefore, we investigated 24 Asians students (12 females) and 24 age-matched European students (12 females) during an explicit emotion recognition task, using Caucasian facial expressions only, on a high-field MRI scanner. Results Analysis of functional data revealed bilateral amygdala activation to emotional expressions in Asian and European subjects. However, in the Asian sample, a stronger response of the amygdala emerged and was paralleled by reduced recognition accuracy, particularly for angry male faces. Moreover, no significant gender difference emerged. We also observed a significant inverse correlation between duration of stay and amygdala activation. Conclusion In this study we investigated the “alien-effect” as an initial problem during cultural assimilation and examined this effect on a behavioral and neural level. This study has revealed bilateral amygdala activation to emotional expressions in Asian and European females and males. In the Asian sample, a stronger response of the amygdala bilaterally was observed and this was paralleled by reduced performance, especially for anger and disgust depicted by male expressions. However, no gender difference occurred. Taken together, while gender exerts only a subtle effect, culture and duration of stay as well as gender of poser are shown to be relevant factors for emotion processing, influencing not only behavioral but also neural responses in female and male immigrants.