Governments have restricted public life during the COVID-19 pandemic, inter alia closing sports facilities and gyms. As regular exercise is essential for health, this study examined the effect of pandemic-related confinements on physical activity (PA) levels. A multinational survey was performed in 14 countries. Times spent in moderate-to-vigorous physical activity (MVPA) as well as in vigorous physical activity only (VPA) were assessed using the Nordic Physical Activity Questionnaire (short form). Data were obtained for leisure and occupational PA pre- and during restrictions. Compliance with PA guidelines was calculated based on the recommendations of the World Health Organization (WHO). In total, n = 13,503 respondents (39 ± 15 years, 59% females) were surveyed. Compared to pre-restrictions, overall self-reported PA declined by 41% (MVPA) and 42.2% (VPA). Reductions were higher for occupational vs. leisure time, young and old vs. middle-aged persons, previously more active vs. less active individuals, but similar between men and women. Compared to pre-pandemic, compliance with WHO guidelines decreased from 80.9% (95% CI: 80.3–81.7) to 62.5% (95% CI: 61.6–63.3). Results suggest PA levels have substantially decreased globally during the COVID-19 pandemic. Key stakeholders should consider strategies to mitigate loss in PA in order to preserve health during the pandemic.