The eating disorder screener, Eating Attitudes Test (EAT), has been used widely; however, its usability in specific dietary groups such as vegetarians and vegans remain unknown. Considering the rising popularity of vegetarianism and veganism, the current study aimed to assess the psychometric properties and theoretical assumptions of the 26-item EAT in separate groups of vegetarians (n = 278), vegans (n = 580), and omnivores (n = 413). Confirmatory factor analysis of four models from previous literature revealed inadequate fit of the data, with the exception of a 16-item four-factor model in vegetarians and vegans. Further assessment of the original three-factor model and 16-item four-factor model demonstrated poor psychometric properties. The primarily inadequate test–retest reliability discovered in this study, independent of whether a shortened version was used, raises concerns around the utility and stability of the EAT-26 in vegetarians and vegans. Future research should potentially investigate novel ways of measuring eating disorder pathology in these groups.