There is considerable evidence that receptive language processing depends, at least in part, on cortical tissue in the temporal lobe. Few studies, however, have investigated the neural basis of phonemic discrimination while controlling for non-linguistic and attentional components. In this study nine healthy volunteers discriminated sets of phonemes and tones interspersed by rest periods in a block design paradigm while undergoing fMRI. A comparison of phonemic discrimination with tone discrimination revealed pure left lateralized superior and middle temporal gyrus activations. A comparison of phonemic discrimination with the rest blocks resulted in areas of activations encompassing the language and non-language components of the task. Our findings are consistent with prior reports demonstrating the involvement of the superior and middle temporal gyrus in phonemic discrimination and stronger left temporal lobe activation during speech compared to non-speech sounds. This study also demonstrates the advantage of tone discrimination control blocks as opposed to conventional rest periods to isolate the pure language component of phonemic discrimination.