Objectives: To map the regional patterns of white matter (WM) microstructural abnormalities and gray matter (GM) atrophy exclusively associated with reduced performance in the Symbol Digit Modalities Test (SDMT) and Paced Auditory Serial Addition Test (PASAT) in relapsing-remitting (RR) multiple sclerosis (MS) patients. Methods: In all, 177 RRMS patients and 80 healthy controls (HC) were studied. WM microstructural abnormalities were investigated on diffusion tensor images using tract-based spatial statistics analysis, and regional GM atrophy was estimated on three-dimensional (3D) T1-weighted images using voxel-based morphometry. Results: Compared to HC, RRMS patients showed the expected pattern of cortical–subcortical GM atrophy and WM microstructural abnormalities. In patients, diffusivity abnormalities of supratentorial WM tracts correlated with both SDMT and PASAT scores. Lower SDMT performance was also associated with WM damage in several infratentorial WM tracts. Lower SDMT scores correlated with atrophy of the right anterior cingulate cortex, left postcentral gyrus, and right middle temporal gyrus, whereas lower PASAT scores correlated with atrophy of the deep GM nuclei, bilaterally, and several fronto-temporo-occipital regions. Conclusion: In RRMS patients, regional damage of different neural systems helps explaining reduced performance in SDMT and PASAT. WM microstructural damage typified reduced SDMT performance, whereas atrophy of several GM regions distinguished reduced PASAT performance.