fMRI-based adaptation paradigms (fMR-A) have been used to infer neuronal stimulus selectivities in humans. Inferring neuronal selectivities from fMR-A, however, requires an understanding of the relationship between the stimulus selectivity of neuronal adaptation and responses. We studied this relationship by recording single cells in macaque inferior temporal (IT) cortex, an area that shows fMRI adaptation. Repetition of identical object images reduced the responsiveness of single IT neurons. Presentation of an image to which the neuron was unresponsive did not alter the response to a subsequent image that activated the neuron. Successive presentation of two different images to which the neuron responded similarly produced adaptation, but less so than the repeated presentation of an image. The neuronal adaptation at the single-cell level showed a greater degree of stimulus selectivity than the responses. This complicates the interpretation of fMR-A paradigms when inferring neuronal selectivity.