1. Introduction

A key research question in environmental epidemiology is how weather—and by extension climate—affects infectious diseases [1]. Mechanistically, weather effects are supported by experimental evidence showing that variables such as temperature and relative humidity can sensitively alter the survival time, infectivity, or virulence of many pathogens. These include, among others, respiratory viruses such as SARS-CoV-2 [2] or influenza [3], bacteria such as Streptococcus pneumoniae [4] or Vibrio cholerae [5], and malaria parasites [6]. Beyond its effects on the pathogen, weather can also impact transmission and infection through other mechanisms operating at different scales, from within-host (e.g., low relative humidity impairing mucociliary clearance of influenza A viruses in mice [7]) to between-host effects (e.g., temperature-related seasonal variations in human contact patterns [8]).

Time-series regression (TSR) models are a common statistical approach for studying the effects of weather on infectious disease. In these designs, counts or rates of new cases are typically modeled as a function of weather variables, often using generalized linear models or more flexible alternatives such as generalized additive models. Causal inference tools like directed acyclic graphs (DAGs) can guide model specification by explicitly identifying the exposure, outcome, and appropriate adjustment set [9], thereby reducing bias. Beyond the weather variables of primary interest, TSR formulations commonly include additional covariates intended to account for other drivers of incidence, such as autocorrelation terms to model transmission effects or smooth functions of time to model changes in the immune population [10]. With this approach, the time series of observed incident cases is regressed against weather variables along with other covariates, and the resulting regression coefficients are often interpreted as estimates of the effect of weather on transmission or incidence [11–26] (Note that we use the term effect here, because, despite the use of associational language, the goal of these studies is often causal [27,28]). TSR models are attractive because they are easy to implement and can accommodate nonlinear exposure–response relationships. Consequently, these models have been widely used to study associations between weather and diseases such as influenza, dengue, cholera, hand-foot-and-mouth disease, enteric infections, and COVID-19 [10, 29–52].

Although TSR models provide a practical framework for studying the effects of weather on infectious diseases, several methodological challenges complicate population-level inference [1,10,29]. First, given the observational nature of these data, careful analysis is necessary to distinguish correlation from causation. Second, surveillance typically detects only a subset of cases, leading to underreporting and measurement error. Third, the infection process underlying the observations is latent and dynamic; key variables, such as the number of susceptible or immune individuals, are unobserved but may fluctuate rapidly and strongly influence transmission dynamics [1]. Altogether, these challenges may complicate the use of standard statistical methods that do not explicitly account for the dynamical nature of disease transmission [53].

Despite the accessibility and widespread adoption of TSR models in environmental epidemiology, their reliability remains unclear. In our previous work, we argued that TSR estimates of the weather effects may be subject to measurement bias [27]. As shown in Fig 1, this bias could arise from systematic differences between the unobserved target endpoint, directly affected by weather variables (i.e., the transmission rate), and the observed endpoint (i.e., the incidence rate). A key issue is that two unobserved variables—namely, the numbers of individuals infected and susceptible to infection—affect the causal path linking these two endpoints in a nonlinear, time-varying manner. Theoretically, additional covariates could account for these variables, thereby mitigating this bias. Still, it remains unclear whether such additions work well in practice. Here, we build on this prior work to systematically evaluate the performance of TSR models on case-count data at the population-level to estimate how weather affects infectious diseases. Using an extensive simulation study based on a concise transmission model, we aimed to evaluate the accuracy and precision of TSR estimates compared to a fixed ground truth. We repeated this procedure to quantify changes in TSR performance across varying experimental treatments [54], including climate, pathogen characteristics, observation and process noise levels, and extrinsic seasonal drivers.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Causal directed acyclic graph for the transmission model.

Green boxes represent the observable climatic variables. Dashed boxes show the unobserved states of the system. and are temperature and relative humidity respectively, represents the number of susceptible individuals, is the transmission rate, is the weekly number of new observed cases, and is the total number of cases. Given the chosen transmission model, the latter variable is approximately equal to the total number of infected individuals..

https://doi.org/10.1371/journal.pcbi.1014797.g001

2. Methods

2.1. Overview & modeling workflow

To assess TSR's performance in this context, we conducted a simulation study. Our modeling workflow is illustrated in Fig 2. First, we designed a parsimonious epidemic model in which real-world weather data modulate the transmission rate (see Fig 1 for the directed acyclic graph (DAG) underlying the epidemic model). We then used this model to simulate time series of weekly reported cases, an outcome comparable to that used in TSR studies. Finally, we fitted different TSR models to these time series and assessed their ability to capture the real effects of weather variables on transmission. For comprehensiveness, we repeated this process for weather data across four different climates and under various assumptions about pathogens’ transmissibility and the rate of waning of infection-derived immunity. The steps of the modeling workflow are detailed below.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Workflow of the simulation study.

a) Time series of temperature and relative humidity measurements. b) Set of sampled values of the mean age at first infection and immunity waning rate fed to the causal model. c) DAG of the causal model, here defined as an SIRS transmission model. d) 100 replicate time series are generated for each set of sampled parameters. e) TSR models are fitted to each simulated time series. f) The accuracy and precision of estimates are evaluated in comparison to the ground-truth values of parameters..

https://doi.org/10.1371/journal.pcbi.1014797.g002

2.3. Model formulation

2.3.1. Transmission rate & Weather effects.

To incorporate weather effects into our model, we assumed that temperature () and relative humidity () each directly affected the transmission rate (). This assumption is supported by experimental evidence showing that both weather variables affect the survival of respiratory viruses [2,3,58–61]. Specifically, the transmission rate was modeled by the following equation:

(Eq. 1)

Here, represents the mean-centered temperature at time and the mean-centered relative humidity at time , where and are the temporal averages across the entire time series. The parameter represents the mean transmission rate, defined as , where is the basic reproduction number, is the birth and death rate, and the generation rate (for a complete description of all model parameters, refer to Table B in S1 Appendix). Given the causal design (Fig 1), the parameters and quantify the direct causal effects of temperature and relative humidity on transmission rate.

2.3.2. Transmission model.

Following Ref [27], we formulated a discrete-time SIRS model with demography. We assumed a fixed time step and a generation time of one week, thereby enabling us to specify a control TSR model that could recover the true causal effects of weather if all model variables were perfectly observed (see the “Control” model section below). Given these assumptions, the model was represented by the following set of difference equations:

where the parameter N is the population size, the birth/death rate (assumed equal to maintain a constant population over time), and the average duration of infection-derived immunity. The force of infection is defined as , where denotes the inflow of a small number of infected individuals introduced to prevent epidemic extinction. Finally, is the total number of new cases every week (total incidence rate). For added biological realism, we introduced stochasticity in the transmission rate: at each time step, we multiplied by a random deviate from a Gamma distribution () with mean 1 and standard deviation . Hence, the parameter modulates the magnitude of stochasticity in transmission, and in the limit the transmission model is entirely deterministic. Altogether, the force of infection was modeled as:

2.3.3. Observation model.

To align with the outcomes analyzed in TSR studies, we defined the weekly time series of new observed cases (i.e., the observed incidence rate, denoted by ) as the primary outcome of the model. We generated this outcome using a Negative-Binomial observation model:

with mean and variance . Here, the parameter is defined as the reporting probability and the reporting overdispersion. Hence, this observation model enabled us to generate measurement noise in the simulated data, as expected in real-world data settings. In a baseline parameterization, we set to 0.1, corresponding to a small coefficient of variation for the observation model. We note that this value is lower than the minimum estimated by transmission models across a range of pathogens (see illustrative values from a non-exhaustive search in Table C in S1 Appendix).

2.4. Model parameterization

2.4.1. Parameterization of pathogens’ characteristics.

To make our analysis more generalizable, we considered a range of pathogens with different average durations of immunity (parameter , range considered: 1–20 years) and mean age at first infection (parameter A, range considered: 1–10 years). For our model, the latter parameter is defined as:

Specifically, we used Latin hypercube sampling (over the parameter ranges indicated above) to define 50 synthetic pathogens with distinct pairs of parameters (Fig B in S1 Appendix). We chose these specific parameters for their clear biological interpretation.

2.4.2. Calibration of weather effects on transmission.

To derive realistic values for the weather effect parameters ( and ), we calibrated them to reproduce the seasonal variations in transmission rates estimated in previous modeling studies. Specifically, we considered two modeling studies: one that estimated a ± 15% seasonal amplitude (relative to the mean) in influenza transmission in Tel Aviv, Israel [62], and another that estimated a ± 37.5% seasonal amplitude in RSV transmission in Turku, Finland [63]. Using the observed weather data from both locations during the corresponding study periods and assuming that , we estimated the weather parameter that yielded the best match in relative amplitude between the simulated and empirical transmission rates. The resulting estimates were approximated to in Tel Aviv and in Turku, and we considered both values in subsequent analyses.

2.4.3. Model versions and alternative model formulations.

We defined a baseline version of the transmission model as one with no process noise (), low observation noise (), and the strongest weather effects (). We tested five other model versions by varying different parameters:

1. Increased observation noise ()
2. Addition of process noise ()
3. Decreased weather effects ()
4. Addition of term-time forcing in transmission
5. Addition of lagged effects of weather.

For the latter, we formulated a model that allows weather to affect transmissibility both in the current week and with a one-week lag (see Sect. A in the S1 Appendix).

We also formulated a model variant with an extra source of seasonality in transmission: seasonal variations in host contacts, driven by the alternation of school terms (with higher contact rates) and school holidays (with lower contact rates). This effect was modeled as a multiplicative change in the transmission rate, proportional to during school holidays (defined here as Christmas and summer vacations, weeks 52 and 24–35) and during school terms. Here, represents the amplitude of term-time forcing in transmission, fixed under the assumption that children make 30% fewer contacts during school holidays [64–67]. Accordingly, for time series including term-time forcing, the regression models included a holiday indicator as a covariate, indicating whether the observation fell during a school holiday.

2.5. Regression models formulation and fitting

The TSR models were formulated as Generalized Additive Models (GAMs) with a Negative Binomial distribution to match the observation model. Although GAMs are not universally used in TSR studies, we considered this class of regression models for their flexibility and versatility, particularly for capturing continuous temporal variation while preventing overfitting. We note that many TSR models used in the literature (e.g.[31–36]) are just special cases of GAMs. For each simulated time series of observed incidence, we formulated and fitted two GAMs, as described below.

2.5.1. Control model.

Following Ref [27]., we first identified a suitable TSR model to apply to the outputs of our transmission model. To do so, we log-transformed the total incidence rate:

where is a constant. This equation, together with the NB observation model for the observed incidence rate, indicates that an exact TSR model in our setting is a NB model with a log-link and three covariates: , , and . This model included latent variables and therefore represented an idealized setting in which these are observed. Hereafter, we refer to this model as the “Control” model, as we expect it to recover the true causal effects of weather. We note that this model is conceptually equivalent to the Time-series Susceptible-Infected-Recovered (TSIR) model, a regression-based approach where the reconstruction of the susceptible population is informed by a transmission model [68].

2.5.2. Test model.

In real-world settings, the variables and are unobserved and must be approximated by adding covariates to the TSR model. Hence, we considered a TSR test model with two additional covariates. The first was an autoregressive term—. This addition was based on the DAG (Fig 1) and the fact that, in our parsimonious model, . Hence, the autocorrelation term aims to capture temporal variations in the number of infected individuals. More broadly, including an autoregressive term is generally recommended in TSR models to account for the autocorrelation in the outcome time series [29].

The second additional covariate is a smooth function of time, modeled using a thin plate regression spline. This choice follows the recommendations for the TSR model design [10,29,69] and reflects the practice of multiple TSR studies [34–43]. We set the maximum basis dimension to 52 (approximately one degree of freedom per 10 data points) to provide this covariate with sufficient flexibility to capture temporal variation in the unobserved variable . This choice was based on preliminary analyses showing that the effective degrees of freedom estimated for this smooth tended to saturate at this value. However, we tested a higher value of 75 in a sensitivity analysis.

2.5.3. TSR performance assessment.

To assess each TSR model’s ability to capture the causal effects of weather, we computed two average performance metrics for each combination of weather variable, scenario, and location:

1. Relative Mean Absolute Bias (RMAB). We first calculated the Mean Absolute Bias (MAB) as the absolute difference between the estimated and true weather effects, averaged across within-scenario replicates. The RMAB was then defined as the scenario-specific MAB divided by the absolute value of the true weather effect. For interpretation, an RMAB of 20% means that estimates were on average 20% off the actual weather effect value. Hence, the RMAB quantifies the average estimation accuracy.
2. Relative Mean Standard Error (RMSE). This metric was calculated to allow comparison in precision across tested model versions. For each scenario, we first averaged the standard errors of the estimates across replicates to obtain the Mean Standard Error (MSE). Then, we divided each MSE by the absolute value of the true weather effect. For interpretation, an RMSE of 10% means that the average estimation uncertainty was ± 10% relative to the true weather effect.

With the definitions above, lower RMAB indicates higher accuracy and lower RMSE indicates higher precision.

In addition, we evaluated each metric's standard deviation across scenarios (denoted by ) and replicates (denoted by ) within each scenario; see Sect. B in S1 Appendix for the corresponding equations.

Finally, we also calculated the mean power for every scenario. This was defined as the average across replicates of a binary variable equal to 1 if the sign of the weather effect was correctly inferred (i.e., if the entire 95% confidence interval contained negative values) and 0 otherwise. Hence, this less stringent metric assessed the TSR models’ ability to correctly recover the direction of the weather effect.

2.6. Simulation protocol

In every city and for every parameter pair (representing the pathogen’s characteristics), we simulated 100 stochastic replicates (Fig 3). We then recorded the simulated time series of the observed cases and fitted the formulated TSR models using these time series as the outcome. All transmission models were initialized at their steady state without seasonal forcing.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Simulated time series of the total incidence rate.

The colored lines correspond to three selected scenarios, highlighted to illustrate the different epidemic periodicities, while all remaining simulations are shown in grey. These simulations refer to the baseline parameter set .

https://doi.org/10.1371/journal.pcbi.1014797.g003

3. Results

We first verified that the control model produced accurate and precise estimates of weather effects, with both RMAB and RMSE around 10% at all locations except Rio de Janeiro, with an average RMAB of 17% and average RMSE of 21% across weather variables (Table D in S1 Appendix, Figs 4–5). In addition, all control models had 100% power to detect the correct sign of both weather variables.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. Assessment of TSR models’ performance in estimating the effect of temperature.

Each point represents a tested synthetic pathogen, with TSR performance quantified by the Relative Mean Absolute Bias (RMAB; x-axis, averaged across replicates) and the Relative Mean Standard Error (RMSE; y-axis, averaged across replicates). The color scale represents the scenario-averaged power, defined as the probability that the model correctly recovers the sign of the true effect. The grey points indicate the control model's performance. The parameters used to generate the fitted simulations were: .

https://doi.org/10.1371/journal.pcbi.1014797.g004

Download:

• PNG
  larger image
• TIFF
  original image

Fig 5. Assessment of TSR models’ performance in estimating the effect of relative humidity.

Each point represents a tested synthetic pathogen, with TSR performance quantified by the Relative Mean Absolute Bias (RMAB; x-axis, averaged across replicates) and the Relative Mean Standard Error (RMSE; y-axis, averaged across replicates). The color scale represents the scenario-averaged power, defined as the probability that the model correctly recovers the sign of the true effect. The grey points indicate the control model's performance. The parameters used to generate the fitted simulations were: .

https://doi.org/10.1371/journal.pcbi.1014797.g005

The estimation results for the baseline test model formulation are presented in Figs 4 and 5. Overall, the temperature effect estimates were inaccurate across all locations, with the mean RMAB ranging from 32% in Toronto to 57% in Rio de Janeiro. Furthermore, there was substantial RMAB variability across scenarios ( range across locations: 11–16%) and across stochastic replicates within scenarios ( range across locations: 22–42%). In addition, there was marked uncertainty in the estimates, with mean RMSE ranging from 29% in Rome to 68% in Rio de Janeiro. The RMSE varied among ( range: 6–12%) and within ( range: 2–5%) scenarios. Finally, average statistical power was low, ranging from 78% in Toronto to 35% in Rio de Janeiro.

The estimates for relative humidity were more accurate (mean RMAB range: 15% in Rome to 31% in Rio de Janeiro) and precise (mean RMSE range: 11% in Rome to 20% in Rio de Janeiro), though this performance remains arguably poor given our simplified simulation and estimation protocol. The estimation of relative humidity effects had higher power, ranging from 93% in Rio de Janeiro to 100% in Dubai and Rome. There was also variability in accuracy across ( range for the RMAB: 6–10%) and within ( range for the RMAB: 10–17%) scenarios, although it was less pronounced than for temperature effects (Table E in S1 Appendix). For both weather variables, TSR performance varied across the simulated pathogens’ characteristics, though no consistent association was found (Figs C–D in S1 Appendix). The higher performance for relative humidity than for temperature likely reflects the greater variability of relative humidity and its effect on transmission across the four climates considered (Figs E–F in S1 Appendix). Consistent with this, the tropical location, where temperature variability was lowest (Table A in S1 Appendix), also showed the poorest performance for temperature effect estimates.

The estimation performance for the other model versions is shown in Figs G–K and Tables F–I in S1 Appendix. Increasing the observation noise led to more inaccurate and imprecise estimates (with all values reported as mean ± SD) at each location (Fig G in S1 Appendix), for both temperature (mean RMAB increase of 8% ± 5%, mean RMSE increase of 9% ± 3%) and relative humidity (mean RMAB increase of 6% ± 4%, mean RMSE increase of 3% ± 1%). Similarly, decreasing the effects of weather generally led to worse performance (Fig H in S1 Appendix) (mean RMAB increase of 10% ± 25%, mean RMSE increase of 7% ± 16% for temperature; mean RMAB increase of 10% ± 12%, mean RMSE increase of 5% ± 3% for relative humidity). In contrast, adding process noise or term-time forcing did not, on average, worsen model performance (Fig I in S1 Appendix) (mean RMAB change of 0% ± 9%, mean RMSE change of 0% ± 4% for temperature; mean RMAB decrease of 1% ± 4%, mean RMSE change of 0% ± 1% for relative humidity). However, greater variability across scenarios was observed with term-time forcing, with higher bias in some cases (Fig J in S1 Appendix) (mean RMAB increase of 1% ± 20%, mean RMSE increase of 3% ± 13% for temperature; mean RMAB increase of 1% ± 10%, mean RMSE increase of 1% ± 4% for relative humidity).

In a sensitivity analysis, we found that changing the maximum dimension of the time smooth did not affect the estimates, with average differences in performance of 0% and 7% in RMAB and 1% and 0% for RMSE, respectively, for temperature and relative humidity across locations (Fig K in S1 Appendix).

Finally, we extended the transmission model to allow both weather variables to influence transmissibility in the current week and with a one-week lag. Correspondingly, we modified the statistical formulation of the time series regression (TSR) models to account for these effects. Using the location where TSR had previously performed best (i.e., Toronto, Canada), we generated new data and fitted the adjusted TSR models. After introducing this additional layer of complexity, TSR model performance degraded. Across the effects of the two variables and the two temporal resolutions of their effects, RMAB values ranged from 18% to 224%, and RMSE values from 28% to 264% (Fig L in S1 appendix).

4. Discussion

The main goal of this study was to evaluate the reliability of TSR models in estimating the effects of weather on infectious diseases from case-count data at the population level. Through a comprehensive simulation study based on a parsimonious transmission model, we systematically assessed the accuracy and precision of TSR model formulations across four climates and fifty synthetic pathogens. Overall, we found that TSR generally performed poorly in our simulated system, with high bias and low precision in the effect estimates, especially for temperature. More broadly, our findings emphasize the importance of causal thinking and simulation-based experiments in evaluating study designs in infectious disease epidemiology.

Due to its accessibility and applicability, the TSR is highly popular in environmental epidemiology [23–26,31–33,35–52]. However, earlier research identified potential issues with this design, including challenges in modeling temporal changes in the immune population, autocorrelation due to transmission, and overdispersion [10]. In our study, these complexities were included through a causal framework represented by an SIRS model that captured transmission and susceptible depletion, augmented with an overdispersed (negative binomial) observation model. We deliberately used a concise transmission model because it yielded a control TSR model that recovered the true weather effects in the absence of unobserved system states (Table D in S1 Appendix). In contrast, when TSR models were applied without knowledge of the latent epidemiological states—as would be the case in practice—their estimates were substantially biased, indicating that the covariates added to the test TSR models did not recapitulate the latent process well (Tables E–I in S1 Appendix).

This inaccuracy is notable, given our simplified modeling framework—in which a single regression coefficient quantified the effect of each weather variable—and our use of advanced regression models that permit the estimation of flexible, nonlinear functions of time. Hence, our results caution against the indiscriminate use of TSR models for estimating the effects of weather on disease dynamics. TSR models may be more appropriate in other systems, where the state variables affecting the link between exposure and outcome are observed or can be reliably approximated.

Importantly, the TSR test model’s performance may have been artificially inflated in our setting. This is because in our model , due to the fixed generation time, a property that does not hold in the real world, due to distributed generation times. This consideration further reinforces our main point about the potential unreliability of TSR models. More broadly, our findings suggest that the choice of covariates in TSR models should ideally be guided by causal models of the disease’s natural history and other data-generating mechanisms, rather than by more general considerations (e.g., autocorrelation or seasonality [10,29]).

Among all the controlled parameters tested, the level of measurement noise had the greatest impact on TSR model performance in our simulations. Even low values—below the levels reported in the disease modeling literature—significantly reduced the accuracy and precision of the estimates. These findings align with earlier research that highlights measurement noise as a relevant issue in environmental epidemiology [10]. From a theoretical standpoint, we hypothesize this result is explained by high-frequency weather variability and the fact that, for directly transmitted infections (as modeled here), weather effects on transmission are immediate. As a result, the weather signature is characterized by high-frequency variations in the incidence data, and this signal may be lost by even modest levels of observation noise in the presence of unobserved variables.

Other influential control parameters included those describing the pathogen. Indeed, our analysis showed marked variability in TSR performance across the two main control parameters (mean age at infection and average duration of immunity) and the resulting basic reproduction number. This result was consistent with epidemiological theory, as these parameters critically influence transmission dynamics, particularly the amplitude, timing, and periodicity of epidemic peaks [72]. However, we were unable to find any clear link between these characteristics and TSR accuracy (Figs C–D in S1 Appendix). In practice, if these characteristics can be identified with reasonable confidence before analysis, we recommend conducting a simulation study similar to ours to assess, a priori, how TSR might perform for the system under investigation.

Similarly, climatic characteristics affected TSR model performance, with less accurate estimation of temperature effects in the tropical climate considered. However, a more thorough understanding of how climatic characteristics affect the identifiability of weather effects would require consideration of a wider range of climates, which we see as valuable future work.

In contrast, other control parameters had a lower impact on TSR performance. In particular, adding process noise increased the bias only marginally in our application. This result was unexpected because, unlike measurement noise, process noise affects the transmission dynamics at every time point, causing its effects to accumulate and compound over time. Nevertheless, process and measurement noise may be stronger in real-world applications.

We acknowledge several limitations of our study. First, we assumed that only two weather variables affected transmission. This choice was motivated by experimental evidence showing the independent effects of temperature and relative humidity on pathogen survival time, e.g., for influenza and SARS-CoV-2 [2,3]. Yet other causal variables—such as UV radiation [73]—may be biologically plausible, but because of the high correlations among weather variables, including them could make TSR even less reliable.

Second, our intentionally parsimonious transmission model overlooked several real-world complexities, such as the possibility of more complex lagged (due to variability in the generation time among individuals) and non-monotonic (e.g., a U-shaped relationship with relative humidity, as indicated by some experimental evidence [2]) weather effects. In particular, such lagged effects are especially relevant for vector-borne diseases, for which our modeling framework would need to be adapted accordingly. Extensions of GAMs have been developed to model lagged effects (e.g., distributed lag nonlinear models [74]), but we note that these were not needed here because we assumed a fixed lag structure. More broadly, accounting for these additional complexities would arguably make the estimation problem more difficult and could further reduce TSR performance. Tentatively supporting this view, TSR performance deteriorated further in an extended simulation model incorporating an additional lag in the weather effects on transmission (Fig L in the S1 Appendix).

Third, we did not take into account the impact of the time scale used for data aggregation. Our discrete-time transmission model simulated the epidemic dynamics on a weekly basis, without considering values at intermediate time steps. In reality, however, the effects of weather occur over shorter time scales (for example, daily), which may not align with the aggregation of the data. This discrepancy could introduce further complexities, likely degrading the performance of estimation even more. At the same time, because surveillance epidemiological data are typically reported weekly, using weekly aggregation provides a more realistic setting for evaluating TSR models in practice.

Fourth, the outcomes considered here were time-series data on incident cases, but other outcomes can be defined and analyzed using TSR. These include the effective reproduction number, which can be reconstructed from the observed incidence rate and then regressed against weather variables of interest [44,48,73]. TSR performance might improve for this outcome, as the underlying causal diagram involves fewer unobserved variables [27]. However, this advantage could be offset by the smoothing introduced by the reconstruction. Although beyond the scope of this study, future research could investigate this question through a simulation study.

Finally, we considered only one TSR model; however, other formulations have been proposed to model the unobserved variables. For example, instead of the time-smooth covariate used here, Imai et al. suggested adding a covariate representing the count of past cases as a proxy for the immune population [10]. However, this approach would require detailed information on the duration of immunity, including both its average and its full distribution; therefore, its feasibility may be limited if such information is unavailable. Future research could explore whether carefully constructed covariates can improve TSR performance. Still, we note that most TSR applications employed simple covariates similar to those in our analysis [29]. More generally, more causally elaborate formulations may improve TSR models and, given the substantial amounts of data needed for reliable parametrization of transmission models, further research on the TSR design is worthwhile.

In conclusion, our findings indicate that despite its popularity and ease of implementation, the TSR design can be unreliable, even when using flexible regression-based statistical methods. These limitations highlight the complexity of infectious disease dynamics, including nonlinearities, various types of noise, and rapidly changing, yet unobserved, variables. Therefore, alternative methods—such as full transmission models or hybrid approaches, such as the TSIR model—may be better suited to clarifying the effects of weather and, by extension, climate, on infectious diseases. More broadly, our results emphasize the value of simulation studies grounded in causal reasoning about the data-generating process for assessing existing methods under “experimental” conditions.

Supporting Information

Acknowledgments

Computations were performed using the Max Planck Computing and Data Facility (MPCDF) high-performance computing cluster.

Declaration of interests: I have read the journal's policy and the authors of this manuscript have the following competing interests: M.D.d.C reports consulting fees from MSD, GSK, Moderna, and Vaxcyte for work unrelated to this study. All other authors declare no competing interests.