Background: Longitudinal epidemiological studies require both the periodic update of intake information via repeated dietary survey and the minimization of subject burden in responding to questionnaires. We developed a 66-item Food Frequency Questionnaire (short-FFQ) for the Japan Public Health Center-based prospective Study for the Next Generation (JPHC-NEXT) follow-up survey using major foods from the FFQ developed for the original JPHC Study. For the JPHC-NEXT baseline survey, we used a larger 172-item FFQ (long-FFQ), which was also derived from the JPHC-FFQ. We compared the validity of ranking individuals by levels of dietary consumption by these FFQs among residents of selected JPHC-NEXT study areas. Methods: From 2012 to 2013, 240 men and women aged 40–74 years from five areas in the JPHC-NEXT protocol were asked to respond to the long-FFQ and provide 12-day weighed food records (WFR) as reference; 228 also completed the short-FFQ. Spearman’s correlation coefficients (CCs) between estimates from the FFQs and WFR were calculated and corrected for intra-individual variation of the WFR. Results: Median CC values for energy and 53 nutrients for the short-FFQ for men and women were 0.46 and 0.44, respectively. Respective values for the long-FFQ were 0.50 and 0.43. Compared with the long-FFQ, crossclassification into exact plus adjacent quintiles with the short-FFQ ranged from 68% to 91% in men and 58% to 85% in women. Conclusions: Similar to the long-FFQ, the short-FFQ provided reasonably valid measures for ranking middle-aged and elderly Japanese for many nutrients and food groups. The short-FFQ can be used in follow-up surveys in prospective cohort studies aimed at updating diet rank information.