The cortical responses to auditory stimuli undergo rapid and dramatic changes during the first 3 years of life in normally developing (ND) children, with decreases in latency and changes in amplitude in the primary peaks. However, most previous studies have focused on children >3 years of age. The analysis of data from the early stages of development is challenging because the temporal pattern of the evoked responses changes with age (e.g., additional peaks emerge with increasing age) and peak latency decreases with age. This study used the topography of the auditory evoked magnetic field (AEF) to identify the auditory components in ND children between 6 and 68 months (<i>n</i> = 48). The latencies of the peaks in the AEF produced by a tone burst (ISI 2 ± 0.2 s) during sleep decreased with age, consistent with previous reports in awake children. The peak latencies of the AEFs in ND children and children with autism spectrum disorder (ASD) were compared. Previous studies indicate that the latencies of the initial components of the auditory evoked potential (AEP) and the AEF are delayed in children with ASD when compared to age-matched ND children >4 years of age. We speculated whether the AEF latencies decrease with age in children diagnosed with ASD as in ND children, but with uniformly longer latencies before the age of about 4 years. Contrary to this hypothesis, the peak latencies did not decrease with age in the ASD group (24-62 months, <i>n</i> = 16) during sleep (unlike in the age-matched controls), although the mean latencies were longer in the ASD group as in previous studies. These results are consistent with previous studies indicating delays in auditory latencies, and they indicate a different maturational pattern in ASD children and ND children. Longitudinal studies are needed to confirm whether the AEF latencies diverge with age, starting at around 3 years, in these 2 groups of children.