Cognitive deficits have been reported in children who experienced early neglect, especially children raised in institutionalized settings. Previous research suggests that early neglect may differentially affect the directional organization of white matter in the prefrontal cortex (PFC). This may be one mechanism to explain cognitive deficits associated with neglect. To test this idea, properties of white matter and neurocognitive performance were assessed in children who suffered early neglect and those raised in typical environments (n = 63, Mage = 11.75 years). As predicted, prefrontal white matter microstructure was affected, consistent with more diffuse organization, in children that suffered early neglect and this was related to neurocognitive deficits. Such findings underscore how early adversity may affect the PFC and explain cognitive deficits associated with neglect.