In the elderly, walking while simultaneously engaging in other activities becomes more difficult. This study aimed to examine the changes in cortical activity during walking with aging. We try to reveal the effects of an additional task and increased walking speed on cortical activation in the young-old and the old-old elderly. Twenty-seven young-old (70.2 ± 3.0 years) and 23 old-old (78.0 ± 2.3 years) participated in this study. Each subject completed four walking tasks on the treadmill, a 2 × 2 design; two single-task (ST) walking conditions with self-selected walking speed (SSWS) and fast walking speed (FWS), and two dual-task (DT) walking conditions with SSWS and FWS. Functional near-infrared spectroscopy was applied for measurement of cerebral oxyhemoglobin (oxyHb) concentration during walking. Cortical activities were increased during DT conditions compared with ST conditions but decreased during the FWS compared with the SSWS on the primary leg motor cortex, supplementary motor area, and dorsolateral prefrontal cortex in both the young-old and the old-old. These oxyHb concentration changes were significantly less prominent in the old-old than in the young-old. This study demonstrated that changes in cortical activity during dual-task walking are lower in the old-old than in the young-old, reflecting the reduced adaptive plasticity with severe aging.