This was a retrospective cohort study, which aimed to investigate the factors associated with hesitancy to receive a third dose of a coronavirus disease 2019 (COVID-19) vaccine. A paper-based questionnaire survey was administered to all participants. This study included participants who provided answers in the questionnaire about whether they had an intent to receive a third dose of a vaccine. Data on sex, age, area of residence, adverse reactions after the second vaccination, whether the third vaccination was desired, and reasons to accept or hesitate over the booster vaccination were retrieved. Among the 2439 participants, with a mean (±SD) age of 52.6 ± 18.9 years, and a median IgG-S antibody titer of 324.9 (AU/mL), 97.9% of participants indicated their intent to accept a third vaccination dose. The logistic regression revealed that participants of a younger age (OR = 0.98; 95% CI: 0.96–1.00) and with a higher antibody level (OR = 2.52; 95% CI: 1.27–4.99) were positively associated with hesitancy over the third vaccine. The efficacy of the COVID-19 vaccine and concerns about adverse reactions had a significant impact on behavior regarding the third vaccination. A rapid increase in the booster dose rate is needed to control the pandemic, and specific approaches should be taken with these groups that are likely to hesitate over the third vaccine, subsequently increasing booster contact rate.