The study investigated the processes underlying the retrieval of action information about functional object pairs, focusing on the contribution of procedural and semantic knowledge. We further assessed whether the retrieval of action knowledge is affected by task demands and age. The contribution of procedural knowledge was examined by the way objects were selected, specifically whether active objects were selected before passive objects. The contribution of semantic knowledge was examined by manipulating the relation between targets and distracters. A touchscreen-based search task was used testing young, middle-aged, and elderly participants. Participants had to select by touching two targets among distracters using two search tasks. In an explicit action search task, participants had to select two objects which afforded a mutual action (e.g., functional pair: hammer-nail). Implicit affordance perception was tested using a visual color-matching search task; participants had to select two objects with the same colored frame. In both tasks, half of the colored targets also afforded an action. Overall, middle-aged participants performed better than young and elderly participants, specifically in the action task. Across participants in the action task, accuracy was increased when the distracters were semantically unrelated to the functional pair, while the opposite pattern was observed in the color task. This effect was enhanced with increased age. In the action task all participants utilized procedural knowledge, i.e., selected the active object before the passive object. This result supports the dual-route account from vision to action. Semantic knowledge contributed to both the action and the color task, but procedural knowledge associated with the direct route was primarily retrieved when the task was action-relevant. Across the adulthood lifespan, the data show inverted U-shaped effects of age on the retrieval of action knowledge. Age also linearly increased the involvement of the indirect (semantic) route and the integration of information of the direct and the indirect routes in selection processes.