Evaluation of Deep Learning Based Early Warning Indicators of Epidemic Outbreaks

In many natural dynamical systems, tipping points emerge where slowly changing external factors induce shifts from one state to another, known as critical transitions or bifurcations. In epidemic systems, detecting these transitions is important for outbreak preparedness. Various statistical indicators like lag-1 autocorrelation and variance have been employed as early warning signals (EWS) of critical transitions by showing how the dynamics of the system exhibit critical slowing down (CSD) near the transition. More recently, deep learning-based algorithms have been developed that attempt to more accurately detect and classify bifurcations; however, they are limited by the systems simulated in the training dataset, the length of the time series, and have not been rigorously evaluated on real-world epidemic data spanning multiple geographic regions. In this paper, we evaluate existing deep learning-based bifurcation prediction models, on a CDC-curated dataset containing weekly influenza hospital admissions across all fifty U.S. states and the nation as a whole. We develop a pipeline for data preparation, reproduction number (Rt) estimation using the EpyEstim package (a Python implementation of the EpiEstim framework [4]), and bifurcation point identification. We systematically evaluate the models under several windowing strategies, including rolling, expanding, and fixed-length windows, and report accuracy, precision, recall, F1 score, and specificity. Our best-performing configuration (expanding window, length 100) achieves an accuracy of 90.09%, precision of 72.86%, recall of 96.23%, and F1 score of 82.93%. We additionally discuss Markov Regime Switching models as an alternative framework for detecting epidemic transitions. Our findings indicate that these models can generalize to real-world, state-level influenza data and are capable of online/real-time prediction.