Fig 1.
Optimal window selection using APE.
(A) An observed incidence curve (blue dots) is sequentially and causally predicted over time s ≤ t using effective reproduction number estimates based on two possible windows lengths of k1 and k2 (blue shaded). Predictive distributions are summarised by red error bars (shown only for times t1 and t2, respectively for k1 and k2), (B) The true reproduction number (Rs, dashed black) is estimated under each window length as (blue) and
(grey). Large windows (k1) smooth over fluctuations. Small ones (k2) recover more changes but are noisy. (C) The APE assesses k1 and k2 via the log-loss of their sequential predictions (i.e. from red error bars across time). The window with the smaller APE is better supported by this incidence curve. See Methods for more mathematical details.
Fig 2.
Selection for stable and fluctuating epidemics.
Left graphs compare estimates (blue with 95% confidence intervals) at the APE window length k* to those when k is set to its upper and lower limits. Right graphs give corresponding one-step-ahead predictions
given the window of data Iτ(s−1) (blue with 95% prediction intervals). Dashed lines are the true Rs numbers (left) and dots are the true Is counts (right). The panels examine (A) stable (constant) and (B) periodically varying changes in Rs.
Fig 3.
Selection for epidemics with interventions.
Left graphs compare estimates (blue with 95% confidence intervals) at the APE window length k* to those when k is set to its upper or lower limits. Right graphs give corresponding one-step-ahead predictions
given the window of data Iτ(s−1) (blue with 95% prediction intervals). Dashed lines are the true Rs numbers (left) and dots are the true Is counts (right). The panels examine (A) exponentially rising and decaying and (B) piecewise falling Rs due to differing interventions.
Fig 4.
We compare the APE metric (blue, left y axes) to the percentage of true incidence values, Is+1 that fall outside the 95% prediction intervals of (red, right y axes) for various window sizes k. The dashed line is k*. Panels correspond to those of Figs 2 and 3.
Fig 5.
Real-time APE sensitivity to increasing transmission.
We simulate 103 independent epi-curves under renewal models with sharply (A) increasing and (B) recovering epidemics. Top graphs give the true (green) and predicted (blue) incidence ranges, the middle ones provide estimates of Rs under the final and the bottom graphs illustrate how successive
choices from APE vary across time and are sensitive to real-time elevations in transmission.
Fig 6.
Real-time APE sensitivity to rapid epidemic control.
We simulate 103 independent epi-curves under renewal models with (A) one effective and (B) two partially effective interventions. Top graphs give the true (green) and predicted (blue) incidence ranges, the middle ones provide estimates of Rs under the final and the bottom graphs illustrate how successive
choices from APE can reliably and rapidly detect the impact of real-time control actions.
Fig 7.
The successive (in time) APE scores for each window length, k, are shown in grey. The scores for the smallest and largest k are in cyan and magenta respectively. Graphs correspond to the models in Figs 5 and 6. Dashed lines are the change-times of each model. In (5A) the change-time does not significantly affect the epi-curve shape and so the APE scores are close together. In (5B), (6A) and (6B) the change-times notably alter the epi-curve shape so that the choice of k becomes critical to performance.
Fig 8.
Empirical prediction accuracy.
We compare the APE metric (dotted blue, left y axis) to the percentage of true incidence values, Is+1, which fall outside the 95% prediction intervals of (dotted red, right y axis) across the window search space k. The dashed line gives k* (black) and k = 7 (grey). The top graph presents results for the influenza 1918 dataset, while the bottom one is for SARS 2003 data. We find that heuristic weekly windows lead to appreciably larger forecasting error than the APE selections.
Fig 9.
Selection for pandemic influenza (1918).
Left graphs compare estimates (blue with 95% confidence intervals) at optimal APE window length k* to weekly sliding windows (k = 7), which were recommended in [10]. Right graphs give corresponding one-step-ahead predictions
(blue with 95% prediction intervals). Dashed lines are the R = 1 threshold (left) and dots are the true incidence Is (right). Panel (A) directly uses the empirical influenza (1918) data [10] while (B) smooths outliers in the data as in [16].
Fig 10.
Left graphs compare estimates (blue with 95% confidence intervals) at optimal APE window length k* to weekly sliding windows (k = 7), which were recommended in [10]. Right graphs give corresponding one-step-ahead predictions
(blue with 95% prediction intervals). Dashed lines are the R = 1 threshold (left) and dots are the true incidence Is (right). Panel (A) directly uses the empirical SARS (2003) data [10] while (B) smooths outliers (with a 5-day moving average) in the data as in [16].