We examined the gender-specific association between dietary pattern and risk of developing cholesterolemia based on the data from the Korean Genome and Epidemiology study. A total of 7515 individuals aged 40–69 years participated in this study between 2005 and 2010. Dietary intake was assessed by a semi-quantitative food frequency questionnaire. Low HDL cholesterolemia was defined as a plasma HDL-C level <1.04 mmol/L (men) or <1.30 mmol/L (women), and high LDL cholesterolemia was defined as a plasma LDL-C level >3.37 mmol/L. Multivariate Cox proportional hazard models were used to examine the risk for incident cholesterolemia according to dietary pattern score. Four dietary patterns were derived by gender using factor analysis: prudent pattern; coffee, fat, and sweet pattern; whole grain (men) or white rice and noodle (women) pattern; and westernized pattern. A prudent pattern was inversely associated with risk of low HDL cholesterolemia in both men (Hazard ratio (HR) = 0.76, p for trend = 0.0098) and women (HR = 0.78, p for trend = 0.0324), whereas the coffee, fat, and sweet pattern was positively associated with risk of high LDL cholesterolemia in men only (HR = 1.26, p for trend = 0.0254) after adjustment for potential confounders. Specific dietary patterns were associated with risk of developing cholesterolemia suggesting gender differences.