Functional magnetic resonance imaging (fMRI) in patients with temporal lobe epilepsy (TLE) has demonstrated reorganisation of language functions with greater involvement of the non-dominant hemisphere. The structural brain connections supporting this atypical language dominance have not previously been identified. We performed fMRI of language functions and imaging of white matter connections using MR tractography in 14 patients with unilateral TLE and hippocampal sclerosis and 10 controls. Verb generation and reading comprehension paradigms were used to define functional regions which were used to generate starting regions for tractography. Controls and right TLE patients had a left-lateralised pattern of both language-related activations and the associated structural connections. Left TLE patients showed more symmetrical language activations, along with reduced left hemisphere and increased right hemisphere structural connections. Subjects with more lateralised functional activation had also more highly lateralised connecting pathways. We provide evidence for structural reorganisation of white matter tracts that reflects the altered functional language lateralisation in left TLE patients. The combination of fMRI and tractography offers a promising tool for studying the reorganisation of language functions in many neurological conditions and may prove useful in predicting language deficits following temporal lobe surgery.