When visual sensory information is restricted, we often rely on haptic and auditory information to recognize objects. Here we examined how haptic exploration of familiar objects affects neural processing of subsequently presented sounds of objects. Recent studies indicated that oscillatory responses, in particular in the gamma band (30–100 Hz), reflect cross-modal processing, but it is not clear which cortical networks are involved. In this high-density EEG study, we measured gamma-band activity (GBA) in humans performing a haptic-to-auditory priming paradigm. Haptic stimuli served as primes, and sounds of objects as targets. Haptic and auditory stimuli were either semantically congruent or incongruent, and participants were asked to categorize the objects represented by the sounds. Response times were shorter for semantically congruent compared with semantically incongruent inputs. This haptic-to-auditory priming effect was associated with enhanced total power GBA (250–350 ms) for semantically congruent inputs and additional effects of semantic congruency on evoked GBA (50–100 ms). Source reconstruction of total GBA using linear beamforming revealed effects of semantic congruency in the left lateral temporal lobe, possibly reflecting matching of information across modalities. For semantically incongruent inputs, total GBA was enhanced in middle frontal cortices, possibly indicating the processing or detection of conflicting information. Our findings demonstrate that semantic priming by haptic object exploration affects processing of auditory inputs in the lateral temporal lobe and suggest an important role of oscillatory activity for multisensory processing.