Study design

Individual epidemics provide too few trajectories to reliably distinguish between forecasting approaches, and therefore do not offer sufficient statistical power to draw firm conclusions about comparative model performance. For this reason, our study combines both a synthetic and an empirical component. The synthetic component enables a controlled evaluation across many independently generated epidemic trajectories, while the empirical component evaluates performance on real-world outbreak data from London, New York, San Francisco, Madrid, and Barcelona. A schematic of the study is shown in Fig 1.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Schematic of the forecasting framework.

Synthetic MSM sexual contact networks were generated and used to simulate stochastic SEIR epidemics across a range of epidemiologically plausible scenarios. Simulated incidence data, together with daily mpox incidence data from five cities, were used to fit alternative forecasting models. For New York and San Francisco, quasi-Poisson regression was applied to correct incidence reported by date of report for reporting-date biases. Forecasting models incorporating different assumptions about epidemic attenuation were fitted at multiple epidemic time points and evaluated using the Continuous Ranked Probability Score (CRPS) against incidence observed 22–28 days after the forecast cutoff.

https://doi.org/10.1371/journal.pcbi.1013987.g001

In the synthetic analysis, we simulate epidemics across three classes of networks: (i) 30 simple regular networks in which every node has an identical degree (henceforth referred to as’homogeneous’ networks), (ii) 30 simple heterogeneous networks with degree distributions drawn from a truncated power-law, and (iii) 100 realistic MSM-like networks characterised by an overlapping one-time and recurrent partner structure. Epidemics are generated on each network instance and the five forecasting models are then fitted to the resulting epidemic curves to assess comparative performance under distinct structural assumptions.

In the empirical analysis, we apply the same modelling approaches to real-world epidemic curves in each of the five cities, enabling evaluation under realistic conditions with true behavioural, demographic, and reporting patterns. The code used to generate the analyses and figures in this study is publicly available at https://github.com/willgreen93/attenuated-renewal.

Data

Data for the mpox outbreak in London was provided by the United Kingdom Health Security Agency (UKHSA) [24]. This gave us daily incidence by time of symptom onset, and so was considered agnostic to day-of-week effects in reporting. Data for Madrid and Barcelona was sourced from David García-García et al. [25]. This provided a time-series of daily incidence based on the date of onset, less a fixed incubation period of 7 days.

Data for San Francisco is taken from the publicly available surveillance dashboard by the San Francisco Department of Public Health (SFDPH) [26]. The dataset reports incident cases by date of report. Data for New York was sourced from the New York City Department of Health and Mental Hygiene (NYC DOHMC) [27], with cases reported by date of diagnosis. Both datasets are therefore potentially liable to day-of-week reporting artefacts arising from clinic operating hours and administrative processing. We applied a diagnostic check to each series by fitting a generalised linear model with quasi-Poisson errors and a log link (appropriate for overdispersed count data) regressing daily case counts on day of week. Quasi-Poisson errors accommodate the overdispersion typical of routine surveillance data, and the log link imposes a multiplicative structure whereby weekend suppression acts as a constant proportional scaling of true incidence regardless of epidemic magnitude. We apply this correction as a pre-processing step rather than adjusting within the models in order to keep the observation model identical across the five approaches and support comparison of the approach to forecasting latent dynamics. For New York City, day-of-week effects were substantial (S2 Fig, S2 Table). For San Francisco, a similar day-of-week correction was applied but with more limited impact on the resulting epi-curve (S2 Fig, S2 Table). We divided observed daily counts by the corresponding weekday-specific factors to yield an adjusted series preserving the underlying epidemic trend.

For all cities we only use data from the first wave of the outbreaks in 2022, with the forecasting model fit with a start point 7 days prior to the first case.

Network generation

We developed a novel generative framework for synthetic MSM sexual contact networks that incorporates seven structural features empirically documented in this population: heavy-tailed degree distributions, in which a small fraction of individuals account for a disproportionate share of sexual partnerships [1]; a distinction between one-time and recurrent partnerships, which imply different contact frequencies and generation time distributions [12, 13]; sexual role structure and assortativity (insertive, receptive, versatile), which constrains partnership compatibility and influences transmission risk [4]; degree assortativity, governing whether high-degree nodes preferentially connect to one another [18, 19]; spatial mixing [28]; partnership duration, allowing for either one-time or recurrent interactions. The motivation for jointly incorporating these features was that their effects on network topology, and thus on epidemic dynamics, are non-trivial in combination. In particular, the interaction between degree overdispersion and assortativity determines how rapidly high-degree, high-risk nodes are depleted early in an outbreak, which is central to the forecasting problem studied here. Existing generative models for MSM contact networks typically incorporate one or two of these properties [1, 29]; combining all five within a single probabilistic edge-formation framework enabled a comparison of how well forecasting approaches perform across a range of network complexities.

The synthetic component of the analysis includes three classes of sexual contact networks, each reflecting a different level of structural realism. The homogeneous networks consist of 30 simulations in which all networks have the same degree and there is no role structure or assortativity. The homogeneous network was included as a theoretical comparator to isolate the effect of degree heterogeneity, not as a realistic representation of sexual contact patterns. The heterogeneous networks consist of 30 simulations with truncated power-law degree distributions but no recurrent partnerships or explicit role assortativity. Finally, the realistic MSM-like networks consist of 100 simulations parameterised using the seven structural properties described previously. The number of realisations differs across network classes to reflect their respective parameter space dimensionality (Table 1). Homogeneous and heterogeneous networks have fewer free parameters and serve primarily as benchmarks under simplified structural assumptions so 30 realisations are used to characterise model performance in these settings. MSM-like networks are parameterised by many more structural properties, motivating a larger number of realisations (n = 100) to adequately cover the higher-dimensional parameter space and ensure robust comparisons across network configurations.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Parameter ranges sampled from the latin hypercube for each of the 30 homogeneous, 30 heterogeneous and 100 MSM-like networks. Homogeneous networks impose no assortativity or spatial structure; heterogeneous networks include degree heterogeneity but not assortative or role-based structure; MSM-like networks include full assortativity, role structure, and recurrent partnership dynamics. Parameters not applicable to a given network type are left blank. ^arange estimated from [30], ^bbased on distribution of main and concurrent casual partnerships reported [31], ^capproximate reasonable distribution taken from [32, 33].

https://doi.org/10.1371/journal.pcbi.1013987.t001

In the realistic MSM-like simulations, the contact structure is represented as two overlapping networks: a main (recurrent) partner network, where sexual activity occurs with probability p_rec per day, and a one-time partner network in which contacts occur only once over the simulation period. The one-time network was constructed first, with each edge in the one-time network assigned a single contact day drawn uniformly at random over the simulation period. The recurrent network was constructed second, with an edge between two nodes disallowed in the recurrent network if they were already connected in the one-time network.

Nodes were first assigned overall degrees according to the specified degree distribution, and edges were then formed probabilistically. The probability of an edge between nodes i and j was based on the product of four factors: the available degree of j (), degree assortativity (), role compatibility (), and spatial proximity () (equation 1). Edges were assigned iteratively: for each node i, a set of eligible partner nodes j > i was identified based on remaining degree capacity, role compatibility and spatial distance.

In both the one-time and the recurrent network, for each eligible partner, the edge formation probability was computed according to Equation 1, and a set of partners was sampled without replacement weighted by these probabilities. After each connection, the available degrees for both nodes were updated to ensure the final network matched the specified degree sequence.

(1)

The degree of node j, was sampled from a truncated power-law distribution:

(2)

Degree assortativity (see Fig 2) was described by a parameter whereby in the case () there was no preferential mixing between nodes of a similar degree. Increasing resulted in degree assortativity, where nodes connect more preferentially based on the relative difference in the overall number of partners, described by:

(3)

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Examples of plausible networks with 100 nodes sharing the same power-law exponent () and mean degree but with differing levels of degree assortativity () and role assortativity ().

Insertives are blue, receptives orange, versatiles green. (A) Base network with no preferential degree or role mixing; measured degree assortativity is negative as expected for truncated power-law degree distributions. (B) Network with increased degree assortativity (), where high-degree nodes connect preferentially with other high-degree nodes. (C) Network with increased versatile assortativity (), where versatile nodes connect preferentially with other versatiles and receptive/insertive nodes connect preferentially with insertive/receptive nodes.

https://doi.org/10.1371/journal.pcbi.1013987.g002

Spatial proximity was incorporated as a node-level attribute: each node was assigned coordinates drawn uniformly from the unit square, and the probability of edge formation between nodes i and j was modulated by an exponentially decaying spatial kernel parameterised by (equation 4). This reflects the empirical finding that sexual partnerships are more likely to form between geographically proximate individuals [28], thus the spatial parameter influences network topology during edge formation.

(4)

Role assortativity (Fig 2) described the extent to which versatile nodes mix with versatile nodes, or, equivalently, the extent to which insertive/receptive partners seek out receptive/insertive partners respectively. This is composed of a matrix (with columns and rows giving the mixing ratios of insertive, receptive and versatiles respectively) and a tuning parameter . In the extreme cases where is 2, versatiles mix exclusively with versatiles, and receptive/insertive nodes mix exclusively with insertive/receptive nodes respectively. When is −1, versatiles mix exclusively with receptive or insertive nodes, and do not engage in within-group mixing:

(5)

To validate the generative framework, we measured key structural properties directly from the 100 generated MSM-like networks, including the realised power-law exponent, degree assortativity coefficient (), versatile assortativity proportion (, defined as the proportion of edges incident to at least one versatile node that connect two versatile nodes), mean edge distance, and clustering coefficient. Each input parameter varied monotonically and strongly with its realised network property (Spearman correlation coefficient r > 0.90; S1 Fig), confirming that the generator provides control over the target structural features across a broad realised range.

Generated networks were not fully connected; across the 100 MSM-like realisations, the largest connected component contained a median of 86.2% of nodes (IQR: 83.3-88.2%, range: 74.1-93.3%). The second largest component never exceeded 1% of the size of the giant component, indicating that remaining nodes were largely isolated or in very small clusters rather than large competing sub-components. Two features of the epidemic simulation design (detailed below) ensured that epidemics propagated through the giant component in practice. First, the epidemic was seeded probabilistically in nodes with at least 10 connections, which are overwhelmingly located within the giant component given the heavy-tailed degree distribution. Second, simulations were rerun until a minimum of 500 cases were observed, effectively discarding any realisations in which the epidemic failed to propagate beyond a small isolated subcomponent.

Epidemic simulation

Epidemics were simulated using a stochastic SEIR (Susceptible–Exposed–Infectious–Removed) framework on the generated networks, chosen for its generalisability to a broad range of infections. At each discrete time step, the probability that a susceptible individual j acquired infection from their infectious contacts was given by

(6)

where denotes the set of infectious individuals at time t, is the per-act transmission probability determined by sexual role and directionality, and represents the neighbourhood of j. The contact indicator equals 1 if nodes i and j had sexual contact at time t and 0 otherwise. For recurrent partnerships, contact at each time step occurs with probability p_rec; for one-time partnerships, contact occurs only on the single day assigned to that edge during network construction.

The role-stratified transmission matrix was constructed from a baseline transmission probability p_trans and a directionality parameter , which captures the elevated per-act transmission risk associated with the receptive role:

where rows index the infectee role and columns the infector role (rec = receptive, ins = insertive, ver = versatile). The parameterisation ensures that the mean of the insertive-to-receptive and receptive-to-insertive transmission probabilities equals p_trans for all values of d, so that p_trans retains its interpretation as an average per-contact transmission probability. When d = 1 all non-zero entries reduce to p_trans, recovering a role-agnostic transmission model.

In all networks the epidemic was seeded with a single infection. In the heterogeneous and realistic MSM-networks, the index case was chosen probabilistically among individuals with more than ten edges with a probability proportional to the number of edges the individual has. Individuals who become infected entered the exposed compartment, E and remained there for a fixed incubation period before progressing to the infectious state, I. Infectious individuals remain infectious for a period , after which they transitioned to the removed state, R. The allowed parameter ranges for the network generation and epidemic simulation are given in Table 1.

Forecasting approaches

We consider five different approaches for the forecast of daily incidence, I(t), spanning from purely statistical to attenuated renewal processes. This includes one statistical model, a model based on the renewal equation, and three extensions of this model where the reproduction number becomes attenuated over time.

Renewal process models provide a natural framework for short-term epidemic forecasting by expressing current incidence I(t) as a weighted sum of past incidence, with weights given by the generation time distribution, [34]:

(7)

The effective reproduction number summarises current transmission intensity: values above one indicate epidemic growth, values below one indicate decline. Forecasting then reduces to specifying how is expected to evolve beyond the observation window: different assumptions about this evolution define the model comparison in this study.

The log-random walk (LRW) [35] serves as a purely statistical baseline, making no mechanistic assumptions about transmission and instead extrapolating recent incidence trends:

(8)

For the renewal models with susceptible depletion, the daily incidence is generated as a function of the reproduction number R(t); the historic incidence I(t), the population size N and the generation time distribution by equation 9.

(9)

where the term accounts for susceptible depletion under the assumption of homogeneous mixing.

The non-attenuated renewal equation model was based on a random walk in R, with a step-size of 3 days (RRAND) [36]:

(10)

We further use a range of models where the reproduction number is attenuated over time in order to capture susceptible depletion in the network beyond what would be expected under homogeneous mixing. The models considered were as follows.

Biased random walk in R, also using a step size of 3 days [37] (RBIAS). RBIAS extends the random walk model (RRAND) by introducing a systematic downward drift , representing a natural augmentation in which retains the flexibility of a random walk but is subject to a constant pull towards lower values over time [37]. This drift acts as a simple, minimally-parameterised attenuation mechanism: without imposing any functional form on the trajectory of decline, it encodes the prior expectation that transmission intensity tends to decrease as an epidemic progresses, whether through susceptible depletion, behavioural change, or public health intervention.

(11)

A reverse-sigmoidal decay in R as a function of time (SIGT). The reverse-sigmoid functional form is motivated by the two-network structure of MSM sexual contact networks. Early in an epidemic, transmission is driven primarily by the one-time partner network, where high edge overdispersion results in rapid infection of high-degree nodes and a correspondingly high effective reproduction number. As these highly-connected individuals are depleted from the susceptible population, transmission increasingly reflects the recurrent partner network, where connectivity is lower and more homogeneous. The sigmoid was chosen as it captures a smooth transition from the high effective reproduction number characteristic of the early epidemic, driven by the one-time partner network, to the lower reproduction number of the later epidemic, dominated by recurrent partnerships, while yielding directly interpretable parameters. Other monotonically decreasing smooth functional forms, such as the Gompertz decay or generalised logistic, could in principle be considered but are left for future work.

(12)

Exponential decay in R as a function of attack rate (EXPI). Unlike the SIGT model, where attenuation is indexed by time, EXPI links the decline in R over and above homogeneous susceptible depletion also directly to the cumulative proportion of the population infected (). This is motivated by the observation that highly-connected nodes are disproportionately likely to be infected at very low attack rates - even the earliest infections are concentrated among high-degree individuals - meaning high-connectedness depletion begins almost immediately when treated as a function of attack rate rather than calendar time. An exponential functional form, which declines monotonically from without an inflection point, is therefore more appropriate than a sigmoid when attenuation is indexed by attack rate rather than time. The inclusion of a floor parameter R_floor serves the same purpose as in the SIG model, capturing the residual transmission sustained by recurrent partnerships once one-time contact susceptibles are largely exhausted. EXPI therefore serves as a useful comparator to SIGT:

(13)

For all models we use a Negative Binomial observation model for reported incidence:

(14)

where is an overdispersion parameter.

Details of the model parameters and their prior distributions are provided in the supplementary material. (Table 1).

Prior distributions were chosen to be weakly informative, centred on epidemiologically plausible values while allowing the likelihood to dominate inference where data are sufficient (S1 Table). The prior on R₀ was log-normal with median 10 and a 95% interval spanning 4.6 - 21.9, allowing for a broad range of early exponential growth. The overdispersion parameter is centred at 30, but again this was weakly informative with 95% bounds of 11.3 and 79.9, suggesting moderate overdispersion. Priors on attenuation parameters were chosen to permit a broad range of epidemic trajectories, from rapid to gradual decline. Shared parameters (R₀, I(0), ) use identical priors across all renewal models, ensuring that performance differences reflect model structure rather than prior specification.

Fitting approach and evaluation

For the empirical analysis, the generation time distribution was fixed at days, assuming a 7-day incubation period [38] and 14-day infectious period [39], with transmission equally likely across the infectious period. MSM population sizes for each city were fixed from external estimates (S1 Table). In the synthetic analysis, both the network size N and generation time distribution were assumed to be known exactly, taken directly from the simulation parameters used to generate each epidemic. A complete list of fixed inputs and estimated parameters, presented separately for the synthetic and empirical analyses, is provided in the supplementary material (S1 Table).

For each simulation, forecasts were generated at ten equally spaced time points spanning the period from when 5% to 95% of all infections had occurred, corresponding to a median inter-forecast interval of 8.5 days (95% interval across simulations: 4.1-16.8 days). This spacing ensured that forecast origins were distributed across the full epidemic trajectory - encompassing growth, peak, and decline - rather than concentrated in any single phase. Each model predicted incidence 28 days into the future from each forecast cutoff, a horizon chosen to reflect the timescale over which public health authorities must plan resource allocation and intervention responses, and consistent with established infectious disease forecasting frameworks [40, 41]. For each forecast, the predicted and observed daily incidence were summed to weekly incidence, and the CRPS was computed on this weekly total. Model performance was assessed based on predicted incidence during days 22–28 post-cutoff (the final week of the horizon) rather than the full window. Near-term predictions (days 1–7) tend to be accurate across all models as they extrapolate from data close to the forecast cutoff, making early-horizon performance uninformative for model discrimination; longer-horizon accuracy is the more operationally relevant quantity, where decisions must be made well in advance of projected peaks or troughs. For completeness, mean CRPS decomposed by individual forecast week (days 1–7, 8–14, 15–21) is provided in the supplementary material (S3-S5 Tables).

Models were fit using Hamiltonian Monte Carlo (HMC) as implemented in the probabilistic programming language Stan [42]. For each simulation we initially ran four chains with 500 warm-up steps and 500 iterations. The number of sampling iterations was increased as needed until all parameters achieved convergence (). Divergent transitions were addressed by incrementally increasing adapt_delta. For a small number of edge-cases (0.2% of the combinations of model, epidemic-phase, and simulation number), divergent iterations or high were not fixed with increasing adapt_delta or iteration count, so these were discounted from the analysis. HMC uses gradient-based methods to efficiently explore high-dimensional parameter spaces, typically achieving faster convergence and better mixing than traditional random-walk Markov Chain Monte Carlo (MCMC) algorithms.

Forecast accuracy was evaluated using the Continuous Ranked Probability Score (CRPS) as implemented in the scoringutils package [43]. For a predictive cumulative distribution function (CDF) F and an observed outcome y, the CRPS is defined as:

(15)

where H is the Heaviside step function. Intuitively, the CRPS integrates the squared distance between the forecast CDF and the empirical CDF at the observed value. The CRPS rewards forecast distributions that place high probability mass close to the observed value: a sharp distribution well-centred on the observation scores well, while a distribution that is poorly centred, unnecessarily wide, or both is penalised. The CRPS therefore jointly assesses calibration (whether observations fall in plausible regions of the forecast distribution), and sharpness (whether the distribution is concentrated).

The CRPS can be decomposed into overprediction, underprediction, and dispersion terms, providing insight into whether model errors arise from systematic bias (forecasts being consistently too high or too low) or from insufficiently calibrated forecast uncertainty (too narrow or too wide predictive intervals).

We computed the Continuous Ranked Probability Score (CRPS) on both the linear and logarithmic scales as recommended in [44]. The linear scale facilitates evaluation of absolute features such as peak incidence or peak demand, while the logarithmic scale provides a measure of relative error, allowing more sensitive assessment of model performance across different epidemic magnitudes.

Simulation studies on synthetic networks

To assess model performance under controlled conditions, we evaluated forecast accuracy across types of network epidemic with parameter combinations drawn by latin hypercube sampling. Model performance was assessed using mean Continuous Ranked Probability Scores (CRPS) on both linear and log-transformed incidence scales. The mean CRPS across each setting, for forecast period from day 22–28, is shown in Table 2. We show the tables for forecast periods 1–7, 8–14 and 15–22 in the Supplementary Material (S3-S5 Tables).

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Forecasting performance for days 22 to 28 across epidemic models. Values represent the mean Continuous Ranked Probability Score (CRPS) for each model across five real-world cities: Barcelona (BCN), London (LON), Madrid (MAD), New York City (NYC), and San Francisco (SF) and three simulated network types: homogeneous, heterogeneous, and MSM-like. Models considered are log random walk (LRW); a renewal model with random walk in (RRAND); a renewal model with a biased random walk in (RBIAS); a renewal model with exponential attenuation of with attack rate to a (EXPI); and a renewal model with sigmoidal attenuation of through calendar time (SIGT). Lower CRPS values indicate better forecast performance. For each column, the best-performing model within each metric is shown in bold. Note that network size differs by class so CRPS can not be compared across columns.

https://doi.org/10.1371/journal.pcbi.1013987.t002

In the homogeneous setting, the standard renewal model RRAND provided the most accurate forecasts. It achieved a mean CRPS of 86 on the linear scale and 0.13 on the log scale, outperforming the next best model (RBIAS) by 65% and 68% respectively. These findings align with expectations that classical renewal models perform well under assumptions of homogeneous mixing.

However, in the heterogeneous network setting, where transmission was concentrated among highly connected individuals, the performance of RRAND deteriorated sharply. It became the worst-performing model with a linear CRPS of 370 and log CRPS of 0.77. In contrast, the SIG model emerged as the most accurate across both the linear and log scales. This reversal in performance highlights the limitations of unconstrained renewal models in the presence of strong contact heterogeneity.

The SIG model also achieved the best performance in the realistic MSM network simulations. Here, the SIG model outperformed all alternatives. On the log scale it performed similarly to EXPI, but it achieved a 20% better performance on the linear scale. SIGT and EXPI outperformed all other models considered quite considerably.

In the MSM-like networks, an error decomposition by epidemic phase demonstrates the random walk model LRW exhibiting the expected temporal bias — under-predicting early and overpredicting late (Fig 3). The RRAND model overpredicted across all phases, which is expected given R starts higher in overdispersed networks as the epidemic progresses through highly-connected nodes. EXPI and SIGT both show a tendency to under-predict early in the epidemic (phases 1–3). When cumulative incidence is low, the data are weakly informative about the attenuation rate , so the posterior remains close to the prior. This may induce premature attenuation of (through in EXPI and the midpoint T in SIGT). We did not assess sensitivity to these priors. In later epidemic phases, this pattern reverses to a slight overprediction, suggesting that in both models attenuation lags behind the true structural decline in .

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Decomposition of the Continuous Ranked Probability Score (CRPS) for the MSM-like networks into overprediction (red; forecast distribution places excess mass above the observation), dispersion (green; penalty for forecast spread, reflecting uncertainty in the predictive distribution), and underprediction (blue; forecast distribution places excess mass below the observation) for MSM-like network simulations (n = 100).

Results are shown separately for the linear scale (top; y-axis in units of cases) and log scale (bottom; y-axis in units of log cases). The x-axis denotes epidemic phase, where earlier phases correspond to forecasts made during epidemic growth and later phases to forecasts made near or after the epidemic peak. A bar dominated simultaneously by large overprediction and underprediction components indicates a model that is highly variable across simulations - alternately projecting too high and too low - without sufficiently wide forecast intervals to capture this variation. For example, RRAND shows consistently large overprediction on the linear scale, particularly at early epidemic phases, reflecting the tendency of an unconstrained random walk to project continued exponential growth when the true epidemic is declining. By contrast, SIGT produces small bars across most phases, with residual error attributable primarily to dispersion, indicating well-centred forecasts with appropriately calibrated uncertainty rather than systematic directional bias.

https://doi.org/10.1371/journal.pcbi.1013987.g003

Examples of the modelled case reproduction number trajectory against the true case reproduction number trajectories using the sigmoid-attenuated function are shown in Fig 4. The fitted trajectories capture the characteristic sigmoidal decline in observed across simulations, reflecting the temporal delay between epidemic onset and the preferential depletion of high-degree nodes that drives transmission slowdown in overdispersed networks. The attenuation midpoint T allows the model to flexibly encode when this hub-depletion effect transitions from a background process to the dominant driver of decline, accommodating the variation in network structure across simulations.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. Comparison of daily case reproduction number to fitted trajectory using the SIG model for an example subset of epidemic simulations.

Points show case reproduction number in the simulation, line and ribbon show the modelled trajectory with 95% CrI accounting for reverse sigmoid depletion and build-up of population immunity.

https://doi.org/10.1371/journal.pcbi.1013987.g004

Application to real-world epidemics

To evaluate model performance in real settings, each forecasting approach was applied to epidemic data from five major cities: London, New York City, San Francisco, Madrid, and Barcelona. The SIG model again demonstrated consistently strong performance, particularly on the linear scale, where it achieved the best fit in all five cities. On the log scale, it was the top performer in four cities and ranked second in Madrid (Table 2).

The city-level CRPS decomposition (Fig 5) reveals a consistent pattern of overprediction for RRAND and EXPI across all cities, concentrated in early epidemic phases (1–4) when epidemics are before their peak. This reflects the failure of unconstrained or slowly attenuating models to anticipate the structural slowdown in transmission: both models project continued exponential growth during the epidemic peak, generating forecasts that substantially exceed observed incidence. The magnitude of this overprediction was particularly severe in New York City, where RRAND reached a mean linear CRPS in excess of 4,000 cases at peak phases. By contrast, RBIAS and SIGT produced small, dispersion-dominated error bars across all cities and phases, indicating well-calibrated forecasts. The average rank of forecasting models across cities and epidemic phases is shown in S3 Fig.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 5. Decomposition of forecast error by epidemic phase and city.

Continuous Ranked Probability Score (CRPS) across epidemic phases (x-axis) for Barcelona (BCN), Madrid (MAD), New York City (NYC), and San Francisco (SF). Each row represents a different city and each column corresponds to a different forecasting model. Stacked bars show the contribution of three CRPS components: overprediction (red), underprediction (green), and dispersion (blue), for each epidemic phase.

https://doi.org/10.1371/journal.pcbi.1013987.g005

A notable divergence between the synthetic and empirical results concerns the EXPI model, which performed competitively in MSM-like simulations but showed substantial overprediction in most cities, particularly in early epidemic phases (Fig 5). In simulated networks, declines systematically with cumulative attack rate as high-degree nodes are mechanistically depleted, providing a clear signal from which the attenuation parameter can be accurately estimated. In real outbreaks, early epidemic dynamics may be dominated by more stochastic fluctuations in case counts, obscuring the relationship between and and leading to a poorly identified and underestimated . The model consequently fails to anticipate the true pace of epidemic decline, producing forecasts that substantially over-predict incidence. Consistent with this explanation, EXPI performance improved markedly at mid- and later- epidemic phases (4–10), once sufficient data had accumulated to better constrain the attenuation parameter.

The SIG model offered superior accuracy across epidemic phases in each city. Visual inspection of city-specific forecasts corroborated these trends, with the sigmoid model performing well overall, but notably overshooting early case trajectories in New York (Fig 6) and undershooting early case counts in Madrid.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 6. City-level forecasts at three epidemic phases for four epidemic models.

Each column corresponds to a city (Barcelona, Madrid, New York City, San Francisco), and each row corresponds to one of four forecasting models (LRW, RRAND, RBIAS, SIGT). Within each panel, model forecasts are shown for three epidemic phases: phase 2 (blue), phase 6 (purple), and phase 10 (pink). Solid black lines represent the median forecast trajectory, while shaded regions denote uncertainty bands. Observed case data are plotted as black points.

https://doi.org/10.1371/journal.pcbi.1013987.g006