Abstract Background Prematurity and perinatal risk factors may influence white matter microstructure. In turn, these maturational changes may influence language development in this high-risk population of children. Objective To evaluate differences in the microstructure of association tracts between preterm and term children and between preterm children with appropriate growth and those with fetal growth restriction and to study whether the diffusion tensor metrics of these tracts correlate with language abilities in schoolchildren with no severe neurological impairment. Materials and methods This study prospectively followed 56 very preterm children (mean gestational age: 28.7 weeks) and 21 age- and gender-matched term children who underwent diffusion tensor imaging at a mean age of 9 years. We used automated probabilistic tractography and measured fractional anisotropy in seven bilateral association tracts known to belong to the white matter language network. Both groups participated in language assessment using five standardised tests at the same age. Results Preterm children had lower fractional anisotropy in the right superior longitudinal fasciculus 1 compared to term children (P < 0.05). Preterm children with fetal growth restriction had lower fractional anisotropy in the left inferior longitudinal fasciculus compared to preterm children with appropriate fetal growth (P < 0.05). Fractional anisotropy in three dorsal tracts and in two dorsal and one ventral tract had a positive correlation with language assessments among preterm children and preterm children with fetal growth restriction, respectively (P < 0.05). Conclusion There were some microstructural differences in language-related tracts between preterm and term children and between preterm children with appropriate and those with restricted fetal growth. Children with better language abilities had a higher fractional anisotropy in distinct white matter tracts.