This study identified the factors associated with current and quitting alcohol drinking in the Oldest Old to better understand the associated factors and mechanisms underlying drinking behaviors in this age group. Results of a questionnaire for drinking behavior in 1015 Japanese Oldest Old citizens aged 85 to 89 years revealed that 56.0% of men and 24.0% of women were current drinkers. A genome-wide association study revealed that the rs671 G > A variation, which corresponds to the aldehyde dehydrogenase 2 (ALDH2) p.E504K missense variant, was significantly associated with current drinking (odds ratio: 3.8, p = 3.33 × 10−31). Variable selection with 41 factors and multivariate regression logistic analysis for current drinking indicated that the rs671 genotype and sex were the most significant factors in the Oldest Old. Further analysis revealed that the rs671 genotype, alcohol-associated biomarkers, a history of heart or kidney disease, and frailty score are factors associated with quitting drinking in the Oldest Old men, whereas smoking history, walking time, and depression score were factors associated with quitting drinking in the Oldest Old women. These results indicate that the ALDH2 p.E504K variation is a major factor associated with current and quitting drinking in the Japanese Oldest Old.