The understanding of empathy from a neuroscientific perspective has recently developed quickly, with numerous functional MRI studies associating different brain regions with different components of empathy. A recent meta-analysis across 40 fMRI studies revealed that affective empathy is most often associated with increased activity in the insula, whereas cognitive empathy is most often associated with activity in the midcingulate cortex and adjacent dorsomedial prefrontal cortex (MCC/dmPFC). To date, however, it remains unclear whether individual differences in brain morphometry in these regions underlie different dispositions in affective and cognitive empathy. In order to test this hypothesis, voxel-based morphometry (VBM) was used to examine the extent to which gray matter density predicts scores from an established empathy measure (Questionnaire of Cognitive and Affective Empathy; QCAE). One hundred and seventy-six participants completed the QCAE and underwent MRI in order to acquire a high-resolution, three-dimensional T1-weighted structural scans. A factor analysis of the questionnaire scores revealed two distinct factors of empathy, affective and cognitive, which confirmed the validity of the QCAE. VBM results revealed gray matter density differences associated with the distinct components of empathy. Higher scores on affective empathy were associated with greater gray matter density in the insula cortex and higher scores of cognitive empathy were associated with greater gray matter density in the MCC/dmPFC. Taken together, these results provide validation for empathy being a multi-component construct, suggesting that affective and cognitive empathy are differentially represented in brain morphometry as well as providing convergent evidence for empathy being represented by different neural and structural correlates.