The association/dissociation of pitch processing between music and language is a long lasting debate. We examined this music-language relationship by investigating to what extent pitch deficits in these two domains were dissociable. We focused on a special neurodevelopmental pitch disorder-congenital amusia, which primarily affects musical pitch processing. Recent research has also revealed lexical tone deficits in speech among amusics. Approximately one-third of Mandarin amusics exhibits behavioural difficulties in lexical tone perception, which is known as tone agnosia. Using mismatch negativities (MMNs), our current work probed lexical tone encoding at the pre-attentive level among the Mandarin amusics with (tone agnosics) and without (pure amusics) behavioural lexical tone deficits compared with age- and IQ-matched controls. Relative to the controls and the pure amusics, the tone agnosics exhibited reduced MMNs specifically in response to lexical tone changes. Their tone-consonant MMNs were intact and similar to those of the other two groups. Moreover, the tone MMN reduction over the left hemisphere was tightly linked to behavioural insensitivity to lexical tone changes. The current study thus provides the first psychophysiological evidence of subgroup differences in lexical tone processing among Mandarin amusics and links amusics' behavioural tone deficits to impaired pre-attentive tone processing. Despite the overall music pitch deficits, the subgroup differences in lexical tone processing in Mandarin-speaking amusics suggest dissociation of pitch deficits between music and speech.