The limbic system is presumed to have a central role in cognitive performance, in particular memory. The purpose of this study was to investigate the relationship between limbic white matter microstructure and neuropsychological function in temporal-lobe epilepsy (TLE) patients using diffusion tensor imaging (DTI). Twenty-one adult TLE patients, including 7 non-lesional (nlTLE) and 14 with unilateral mesial temporal sclerosis (uTLE), were studied with both DTI and hippocampal T2 relaxometry. Correlations were performed between fractional anisotropy (FA) of the bilateral fornix and cingulum, hippocampal T2, neuropsychological tests. Positive correlations were observed in the whole group for the left fornix and processing speed index. In contrast, memory tests did not show significant correlations with DTI findings. Subgroup analysis demonstrated an association between the left fornix and processing speed in nlTLE but not uTLE. No correlations were observed between hippocampal T2 and test scores in either the TLE group as a whole or after subgroup analysis. Our findings suggest that integrity of the left fornix specifically is an important anatomical correlate of cognitive function in TLE patients, in particular patients with nlTLE.