The increase in the elderly population today is a fact. This group of people needs day-to-day care due to their age, and, in addition, they often have health problems. Technology can be used to mitigate these problems. However, it must be borne in mind that most of this population is currently unable to get the most out of electronic devices. To help elders benefit from these devices, systems adapted to their needs, and preferences are needed. In particular, systems that use the elders' contextual information to integrate several aspects of eldercare and adapt them to each elder would provide significant benefits. In this case, the emotions will be used to recognize to what extent an elderly person needs care at certain times of the day and to adapt surrounding IoT systems to their needs and moods. For this purpose, this chapter proposes to use smartphones as the devices that centralize contextual information of the elders, focusing on emotion recognition.