Brain-derived neurotrophic factor (BDNF) plays a key role in learning and memory, but its effects on the fiber architecture of the living brain are unknown. We genotyped 455 healthy adult twins and their non-twin siblings (188 males/267 females; age: 23.7 ± 2.1. years, mean ± SD) and scanned them with high angular resolution diffusion tensor imaging (DTI), to assess how the BDNF Val66Met polymorphism affects white matter microstructure. By applying genetic association analysis to every 3D point in the brain images, we found that the Val-BDNF genetic variant was associated with lower white matter integrity in the splenium of the corpus callosum, left optic radiation, inferior fronto-occipital fasciculus, and superior corona radiata. Normal BDNF variation influenced the association between subjects' performance intellectual ability (as measured by Object Assembly subtest) and fiber integrity (as measured by fractional anisotropy; FA) in the callosal splenium, and pons. BDNF gene may affect the intellectual performance by modulating the white matter development. This combination of genetic association analysis and large-scale diffusion imaging directly relates a specific gene to the fiber microstructure of the living brain and to human intelligence.