Though fetal alcohol syndrome (FAS) and fetal alcohol spectrum disorders are among the most common developmental disorders, their understanding is incomplete. Diffusion tensor imaging (DTI), which is sensitive to microstructural organization in white matter, may provide a relevant measure in this population demonstrating incompletely characterized white matter pathology. In this study, tract-based spatial statistics (TBSS) routine and a skeleton-based region of interest analyses were employed to detect differences in DTI-derived metrics between young adults who were alcohol exposed and an unexposed control group. Participants include 28 with dysmorphic features associated with FAS, 29 who were prenatally exposed but do not show physical effects, and 25 with the same low socioeconomic status but unexposed. The TBSS analysis revealed a statistically significant decrease in fractional anisotropy at the isthmus of the corpus callosum and its connected callosal fibers in dysmorphic individuals relative to controls (clusterwise P(FWE) < 0.05). This finding was consistent with that of the follow-up skeleton-based region of interest analysis (F((2,79)) = 3.256, p = 0.044). In addition, the patterns in axial and radial diffusivity changes suggest that demyelination may be associated with the degraded white matter integrity observed in the dysmorphic group.