Background Coronavirus disease 2019 (COVID-19) has affected all age groups worldwide, but older adults have been affected greatly with an increased risk of severe illness and mortality. Nepal is struggling with the COVID-19 pandemic. The normal life of older adults, one of the vulnerable populations to COVID-19 infection, has been primarily impacted. The current evidence shows that the COVID-19 virus strains are deadly, and non-compliance to standard protocols can have serious consequences, increasing fear among older adults. This study assessed the perceived fear of COVID-19 and associated factors among older adults in eastern Nepal. Methods A cross-sectional study was conducted between July and September 2020 among 847 older adults (≥60 years) residing in three districts of eastern Nepal. Perceived fear of COVID-19 was measured using the seven-item Fear of COVID-19 Scale (FCV-19S). Multivariate logistic regression identified the factors associated with COVID-19 fear. Results The mean score of the FCV-19S was 18.1 (SD = 5.2), and a sizeable proportion of older adults, ranging between 12%-34%, agreed with the seven items of the fear scale. Increasing age, Dalit ethnicity, remoteness to the health facility, and being concerned or overwhelmed with the COVID-19 were associated with greater fear of COVID-19. In contrast, preexisting health conditions were inversely associated with fear. Conclusion Greater fear of the COVID-19 among the older adults in eastern Nepal suggests that during unprecedented times such as the current pandemic, the psychological needs of older adults should be prioritized. Establishing and integrating community-level mental health support as a part of the COVID-19 preparedness and response plan might help to combat COVID-19 fear among them.