The extent of structural injury in sports related concussion (SRC) is central to the course of recovery, long-term effects, and the decision to return to play. In the present longitudinal study, we used diffusion tensor imaging (DTI) to assess white matter (WM) fiber tract integrity within two days, two weeks and two months of concussive injury. Participants were right-handed male varsity contact-sport athletes (20.2 ± 1.0 years of age), with a medically diagnosed SRC (no loss of consciousness). They were compared to right-handed male varsity non-contact-sport athletes serving as controls (19.9 ± 1.7 years). We found significantly increased radial diffusivity (RD) in concussed athletes (n = 12, paired t-test, tract based spatial statistics, p < 0.025) at two days when compared to the two week post injury time point. The increase was found in a cluster of right hemisphere voxels, spanning the posterior limb of the internal capsule (IC), the retrolenticular part of the IC; the inferior longitudinal fasciculus, the inferior fronto-occipital fasciculus (sagittal stratum) and the anterior thalamic radiation. Post hoc, univariate, between group (controls vs. concussed) mixed effects analysis of the cluster showed significantly higher RD at two days (p = 0.002) as compared to the controls; with a trend in the same direction at two months (p = 0.11). Results for fractional anisotropy (FA) in the same cluster showed a similar, but inverted pattern; FA was decreased at two days and at two months post injury when compared to normal controls. At two weeks post injury no statistical differences between concussed and control athletes were found with regard to either RD or FA. These results support the hypothesis of increased RD and reduced FA within 72 hours post injury, followed by recovery that may extend beyond 2 weeks. RD appears to be a sensitive measure of concussive injury.