To investigate associations between dietary patterns and the risk of impaired kidney function, we analyzed data from 14,732 participants (40–89 years) who completed the baseline diet questionnaire of The Fukushima Health Management Survey in 2011. The incidence of chronic kidney disease (CKD) (estimated glomerular filtration rate (eGFR) <60 mL/min/1.73 m2 or proteinuria (≥1+ by dipstick test)) and annual changes in eGFR were assessed from 2012 to 2015. Three major dietary patterns were identified. The adjusted cumulative incidence ratio of the highest vs. lowest tertile of a vegetable diet scores was 0.90 (95% confidence interval (CI): 0.82, 1.00) for eGFR < 60 mL/min/1.73 m2, 0.68 (95% CI: 0.52, 0.90) for proteinuria, and 0.88 (95% CI: 0.80, 0.97) for CKD (P for trend = 0.031, 0.007, and 0.005, respectively). The incident risk of CKD in the highest tertile of juice diet scores was 18% higher than the lowest tertile. The odds ratio of the highest vs. lowest tertile of vegetable diet scores was 0.85 (95% CI: 0.75, 0.98) in the rapidly decreasing eGFR group (P for trend = 0.009). We did not observe significant associations for the meat dietary pattern. A Japanese vegetable diet could reduce the risk of developing impaired kidney function and CKD.