Background Poor cardiorespiratory fitness is a risk factor for cardiovascular morbidity. Alcohol consumption contributes substantially to the burden of disease, but its association with cardiorespiratory fitness is not well described. We examined associations between average alcohol consumption, heavy episodic drinking and cardiorespiratory fitness. Design The design of this study was as a cross-sectional population-based random sample. Methods We analysed data from five independent population-based studies (Study of Health in Pomerania (2008–2012); German Health Interview and Examination Survey (2008–2011); US National Health and Nutrition Examination Survey (NHANES) 1999–2000; NHANES 2001–2002; NHANES 2003–2004) including 7358 men and women aged 20–85 years, free of lung disease or asthma. Cardiorespiratory fitness, quantified by peak oxygen uptake, was assessed using exercise testing. Information regarding average alcohol consumption (ethanol in grams per day (g/d)) and heavy episodic drinking (5+ or 6+ drinks/occasion) was obtained from self-reports. Fractional polynomial regression models were used to determine the best-fitting dose-response relationship. Results Average alcohol consumption displayed an inverted U-type relation with peak oxygen uptake ( p-value<0.0001), after adjustment for age, sex, education, smoking and physical activity. Compared to individuals consuming 10 g/d (moderate consumption), current abstainers and individuals consuming 50 and 60 g/d had significantly lower peak oxygen uptake values (ml/kg/min) (β coefficients = −1.90, β = −0.06, β = −0.31, respectively). Heavy episodic drinking was not associated with peak oxygen uptake. Conclusions Across multiple adult population-based samples, moderate drinkers displayed better fitness than current abstainers and individuals with higher average alcohol consumption.