Although there is evidence that the effect of including a concurrent processing demand on the storage of information in working memory is disproportionately larger for older than younger adults, not all studies show this age-related impairment, and the critical factors responsible for any such impairment remain elusive. Here we assess whether domain overlap between storage and processing activities, and access to semantic representations, are important determinants of performance in a sample of younger and older adults ( N = 119). We developed four versions of a processing task by manipulating the type of stimuli involved (either verbal or non-verbal) and the decision that participants had to make about the stimuli presented on the screen. Participants either had to perform a spatial judgement, in deciding whether the verbal or non-verbal item was presented above or below the centre of the screen, or a semantic judgement, in deciding whether the stimulus refers to something living or not living. The memory task was serial-ordered recall of visually presented letters. The study revealed a large increase in age-related memory differences when concurrent processing was required. These differences were smaller when storage and processing activities both used verbal materials. Dual-task effects on processing were also disproportionate for older adults. Age differences in processing performance appeared larger for tasks requiring spatial decisions rather than semantic decisions. We discuss these findings in relation to three competing frameworks of working memory and the extant literature on cognitive ageing.