Early warning signals (EWSs) aim to predict changes in complex systems from phenomenological signals in time series data. These signals have recently been shown to precede the emergence of disease outbreaks, offering hope that policymakers can make predictive rather than reactive management decisions. Here, using a novel, sequential analysis in combination with daily COVID-19 case data across 24 countries, we suggest that composite EWSs consisting of variance, autocorrelation and skewness can predict nonlinear case increases, but that the predictive ability of these tools varies between waves based upon the degree of critical slowing down present. Our work suggests that in highly monitored disease time series such as COVID-19, EWSs offer the opportunity for policymakers to improve the accuracy of urgent intervention decisions but best characterize hypothesized critical transitions.