Fig 1.
Pairwise Spearman correlation for each pair of variables.
Correlations found to be not significant (p > .05) are not shown.
Fig 2.
Boxplots of quantile scores and MAPE of forecasts from 5 models across all states and months.
Blue points show mean estimate. p-value for Wilcoxon signed rank test on quantile scores: calls/ght=0.64; calls/calls_ght = 0.86; ght/calls_ght = 0.35. p-value for Wilcoxon signed rank test on MAPE: calls/calls_ght = 0.08; ght/calls_ght = 0.49. All other model pairs were statistically significant (p < 1e-4).
Fig 3.
Quantile scores of forecasts from augmented models relative to baseline model (left) and relative to auto model (right), by state. The relative quantile score (RQS), has a range of -100 to 100, a negative value indicating a better forecast than the reference and a positive value indicating a worse forecast. The color lightness (light to dark) represents the magnitude of difference from reference, with a darker shade implying greater improvement.
Fig 4.
Quantile scores of forecasts from augmented models relative to baseline model (left column) and relative to auto model (right column), by year (top row) and month (bottom row). The relative quantile score (RQS), has a range of -100 to 100, a negative value indicating a better forecast than the reference and a positive value indicating a worse forecast. The color lightness (light to dark) represents the magnitude of difference from reference, with a darker shade implying greater improvement.
Fig 5.
Calibration plot of forecasts (top) and hindcasts (bottom). Forecasts from auto (Cramer’s distance [53]=0.03) and to a lesser degree baseline (0.006) appear to be miscalibrated, while the remaining three models have similar and better calibration (5e-4). On the other hand, hindcasts from auto have the best calibration (7e-4) hindcasts and baseline the least calibrated (.015); the augmented models have similar, good calibration (3e-3).
Fig 6.
Quantile scores of forecasts from augmented models relative to baseline model (left) and relative to auto model (right), by state, during the test period (January 2020 – December 2020). Note that forecasts generated for the latter half of the year cannot be fully evaluated until mortality data for 2021 are available (for example, of forecasts generated at August 2020, 5-month ahead (Jan 2021) and 6-month ahead (Feb 2021) could not be evaluated).