Temporal and spectral sound information is processed asymmetrically in the brain with the left-hemisphere showing an advantage for processing the former and the right-hemisphere for the latter. Using monaural sound presentation we demonstrate a context and ability dependent ear-asymmetry in brain measures of temporal change detection. Our measure of temporal processing ability was a gap-detection task quantifying the smallest silent gap in a sound that participants could reliably detect. Our brain measure was the size of the mismatch-negativity (MMN) auditory event-related potential elicited to infrequently presented gap sounds. The MMN indexes discrimination ability and is automatically generated when the brain detects a change in a repeating pattern of sound. MMN was elicited in unattended sequences of infrequent gap-sounds presented among regular no-gap sounds. In Study 1, participants with low gap-detection thresholds (good ability) produced a significantly larger MMN to gap sounds when sequences were presented monaurally to the right-ear than to the left-ear. In Study 2, we not only replicated a right-ear-advantage for MMN in silence in good temporal processors, but also showed that this is reversed to a significant left-ear-advantage for MMN when the same sounds are presented against a background of constant low-level noise. In both studies, poor discriminators showed no ear-advantage, and in Study 2, exhibited no differential sensitivity of the ears to noise. We conclude that these data reveal a context and ability-dependent asymmetry in processing temporal information in non-speech sounds.