The human visual pathways that are specialized for object recognition stretch from lateral occipital cortex (LO) to the ventral surface of the temporal lobe, including the fusiform gyrus. Plasticity in these pathways supports the acquisition of visual expertise, but precisely how training affects the different regions remains unclear. We used functional magnetic resonance imaging to measure neural activity in both LO and the fusiform gyrus in radiologists as they detected abnormalities in chest radiographs. Activity in the right fusiform face area (FFA) correlated with visual expertise, measured as behavioral performance during scanning. In contrast, activity in left LO correlated negatively with expertise, and the amount of LO that responded to radiographs was smaller in experts than in novices. Activity in the FFA and LO correlated negatively in experts, whereas in novices, the 2 regions showed no stable relationship. Together, these results suggest that the FFA becomes more engaged and left LO less engaged in interpreting radiographic images over the course of training. Achieving expert visual performance may involve suppressing existing neural representations while simultaneously developing others.