Multiple sclerosis (MS) is associated with the occurrence of white matter plaques in the central nervous system. These are frequently located in areas interconnecting areas associated with the processing of emotions. Although recent behavioral studies indicated social and affective disturbances in many of these patients, functional studies investigating specific emotional recognition in MS are lacking. We used functional magnetic resonance imaging (fMRI) and lesion mapping in MS-patients to investigate correlates between these measures and emotional facial recognition. Eleven patients whose affective ability was impaired were compared with eleven unimpaired MS-patients and eleven healthy controls (HCs) using a facial expression matching task. Decreased recognition performance was limited to the detection of unpleasant facial expressions (sad, fearful, angry). In evaluating the functional activation maps for the unpleasant facial expressions, we found decreased insular and ventrolateral prefrontal cortex (VLPFC) activation in the impaired group versus the unimpaired groups. We found a close relationship between the inability of solving the task and decreased activation of the left VLPFC and the left anterior insula. In addition, we found a correlation between decreased performance accuracy and the presence of lesions in the left temporal white matter. These data suggest that emotion recognition deficits in MS-patients might be due to the interruption of processing emotionally relevant information, which leads to decreased activation of the VLPFC and the insula.