Young normal-hearing listeners and young-elderly listeners between 55 and 65 years of age, ranging from near-normal hearing to moderate hearing loss, were compared using different speech recognition tasks (consonant recognition in quiet and in noise, and time-compressed sentences) and working memory tasks (serial word recall and digit ordering). The results showed that the group of young-elderly listeners performed worse on both the speech recognition and working memory tasks than the young listeners. However, when pure-tone audiometric thresholds were used as a covariate variable, the significant differences between groups disappeared. These results support the hypothesis that sensory decline in young-elderly listeners seems to be an important factor in explaining the decrease in speech processing and working memory capacity observed at these ages.