Background New research fields to design social robots for older people are emerging. By providing support with communication and social interaction, these robots aim to increase quality of life. Because of the decline in functioning due to cognitive impairment in older people, social robots are regarded as promising, especially for people with dementia. Although study outcomes are hopeful, the quality of studies on the effectiveness of social robots for the elderly is still low due to many methodological limitations. Objective We aimed to review the methodologies used thus far in studies evaluating the feasibility, usability, efficacy, and effectiveness of social robots in clinical and social settings for elderly people, including persons with dementia. Methods Dedicated search strings were developed. Searches in MEDLINE (PubMed), Web of Science, PsycInfo, and CINAHL were performed on August 13, 2020. Results In the 33 included papers, 23 different social robots were investigated for their feasibility, usability, efficacy, and effectiveness. A total of 8 (24.2%) studies included elderly persons in the community, 9 (27.3%) included long-term care facility residents, and 16 (48.5%) included people with dementia. Most of the studies had a single aim, of which 7 (21.2%) focused on efficacy and 7 (21.2%) focused on effectiveness. Moreover, forms of randomized controlled trials were the most applied designs. Feasibility and usability were often studied together in mixed methods or experimental designs and were most often studied in individual interventions. Feasibility was often assessed with the Unified Theory of the Acceptance and Use of Technology model. Efficacy and effectiveness studies used a range of psychosocial and cognitive outcome measures. However, the included studies failed to find significant improvements in quality of life, depression, and cognition. Conclusions This study identified several shortcomings in methodologies used to evaluate social robots, resulting in ambivalent study findings. To improve the quality of these types of studies, efficacy/effectiveness studies will benefit from appropriate randomized controlled trial designs with large sample sizes and individual intervention sessions. Experimental designs might work best for feasibility and usability studies. For each of the 3 goals (efficacy/effectiveness, feasibility, and usability) we also recommend a mixed method of data collection. Multiple interaction sessions running for at least 1 month might aid researchers in drawing significant results and prove the real long-term impact of social robots.