https://orcid.org/0000-0002-7648-7041
Figures
Abstract
Epidemiological delays are key quantities that inform public health policy and clinical practice. They are used as inputs for mathematical and statistical models, which in turn can guide control strategies. In recent work, we found that censoring, right truncation, and dynamical bias were rarely addressed correctly when estimating delays and that these biases were large enough to have knock-on impacts across a large number of use cases. Here, we formulate a checklist of best practices for estimating and reporting epidemiological delays. We also provide a flowchart to guide practitioners based on their data. Our examples are focused on the incubation period and serial interval due to their importance in outbreak response and modeling, but our recommendations are applicable to other delays. The recommendations, which are based on the literature and our experience estimating epidemiological delay distributions during outbreak responses, can help improve the robustness and utility of reported estimates and provide guidance for the evaluation of estimates for downstream use in transmission models or other analyses.
Citation: Charniga K, Park SW, Akhmetzhanov AR, Cori A, Dushoff J, Funk S, et al. (2024) Best practices for estimating and reporting epidemiological delay distributions of infectious diseases. PLoS Comput Biol 20(10): e1012520. https://doi.org/10.1371/journal.pcbi.1012520
Editor: Samuel V. Scarpino, Northeastern University, UNITED STATES OF AMERICA
Published: October 28, 2024
This is an open access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication.
Funding: AC is funded by the National Institute for Health and Care Research (NIHR) Health Protection Research Unit in Modelling and Health Economics, a partnership between the UK Health Security Agency, Imperial College London and LSHTM (grant code NIHR200908); and acknowledges funding from the MRC Centre for Global Infectious Disease Analysis (reference MR/X020258/1), funded by the UK Medical Research Council (MRC). This UK funded award is carried out in the frame of the Global Health EDCTP3 Joint Undertaking. SC acknowledges funding support from the Laboratoire d’Excellence Integrative Biology of Emerging Infectious Diseases program (grant ANR-10-LABX-62-IBEID) and the INCEPTION project (PIA/ANR16-CONV-0005). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
In recent years, an increasing number of real-time data streams have become available for a variety of computational biology applications. For example, NEON [1] is a large-scale observation platform for environmental data in the United States, and EarthRanger [2] is used for wildlife conservation in several countries. These new data streams may improve the ability to perform real-time analyses, such as monitoring biodiversity [3,4] and insect populations [5]. However, delays inherent in these systems (e.g., larval development times for mosquitoes, ecological lags [6]) may pose challenges. The estimation of epidemiological delay distributions, which we discuss in this paper, is a special case of using real-time data to understand the time course of biological processes. Methods for estimating these delays could be broadly applicable to other areas of computational biology.
Epidemiological delay distributions are key quantities for public health policy [7], clinical practice [8], and infectious disease modeling [9–13]. An epidemiological delay is the time between 2 epidemiological events, a primary event and a secondary event. There are numerous examples of delays in infectious disease epidemiology. Three of the most important delays include the incubation period (the time between infection and symptom onset), the serial interval (the time between symptom onset in a given infected person and someone they infect), and the generation interval (the time between infection in a given person and someone they infect). Other commonly used epidemiological delays include hospital lengths of stay and delays from symptom onset to hospitalization, hospitalization to death, and symptom onset to specimen date, among others. Delays vary in length (e.g., between infected individuals) due to biological and epidemiological factors [14–16], such that samples of delays can be characterized by a distribution. These distributions are typically described by one or more parameters. Pair-dependent delays are expected to be more heterogeneous than single-individual delays because they involve more than one person.
Here, we focus on the incubation period and serial interval because they are key inputs in mathematical and statistical models, such as those used for nowcasting/forecasting [9,17] or for scenario-based modeling [18,19], which can guide control strategies. While the incubation period must be positive, the serial interval can be positive or negative [20]. The right tail of the incubation period distribution informs the length of quarantine, while the left tail indicates the earliest time symptoms might develop after infection [8]. By comparing the incubation period and serial interval, we can learn about a pathogen’s tendency for presymptomatic versus symptomatic transmission [21], which can inform the controllability of an epidemic [22]. At the individual level, presymptomatic transmission occurs precisely when the generation interval is shorter than the incubation period of the infector (or, equivalently, when the serial interval is shorter than the incubation period of the infectee). The serial interval is often used as a proxy for the generation interval.
Methods for estimating epidemiological delays have been improving, especially during the COVID-19 pandemic, and recent research has highlighted the importance of appropriately adjusting for 3 statistical issues inherent in the data collection process: censoring, right truncation, and dynamical bias [23–27] (Table 1 and Figs 1 and 2).
The y axis in each panel represents distinct observations of delays. The circles represent primary events, while the squares represent secondary events. The horizontal lines represent the delay between events, while the black vertical lines represent the time at which the data are observed. The arrows represent how cases enter the data set: arrows pointing toward the right indicate that the primary event was observed first, while arrows pointing toward the left indicate that the secondary event was observed first. The brackets “[]” represent interval censoring of the primary and secondary events. Note how primary and secondary events occur in continuous time, while the reporting of events (brackets) always occur at discrete intervals. Delays and events in gray are unobserved. (A) and (B) demonstrate the same scenario, but the direction of the arrows is different. In (A), observation of the delay is based on the primary event, and there is right censoring, while in (B), observation is based on the secondary event, and there is right truncation. (C) Demonstrates an example of a backward distribution in a growing epidemic, when the majority of delays that make up the distribution will be short; hence, the backward distribution will be shorter than the equivalent forward distribution. (D) Demonstrates the reverse in a declining epidemic. Both (C) and (D) show the impact of dynamical bias.
The y axis represents distinct observations of delays. The yellow circles represent the primary events, while the green squares represent the secondary events. The black horizontal lines represent the delay between primary and secondary events, and the vertical dotted lines show the cohorts. The arrows represent how cases enter the dataset: arrows pointing toward the right indicate that the primary event was observed first, while arrows pointing toward the left indicate that the secondary event was observed first. Note that the case ascertainment method does not impact the direction we can cohort the data. For forward cohorts (A and C), all primary events that occurred during the same period are selected and prospectively followed until the secondary event occurs. For backward cohorts (B), all secondary events that occurred during the same period are selected; the timing of the primary events is identified retrospectively. For simplicity, interval censoring has been omitted from this figure; events appear to occur at discrete intervals when in reality, they occur in continuous time.
Examples of diseases for each bias were selected based on convenience (either papers written by this study’s authors or those encountered during the course of their work). Note that not all methods in this table are discussed in the text, and more details can be found in Park and colleagues [23].
Not adjusting for bias can lead to incorrect estimates of delays, which can have direct implications for public health practice. For example, Overton and colleagues found that the mean incubation period for COVID-19 in early 2020 (corresponding to the ancestral strain of SARS-CoV-2) was 3.49 days without adjusting for right truncation compared to 4.69 days when adjusted [25]. Overton and colleagues further showed that the unadjusted estimate would suggest a quarantine of 10 days would capture 99% of cases, compared to a 14-day quarantine for the adjusted estimate [25]. If the unadjusted estimate from this study had been used to inform the length of quarantine, more still-infectious individuals would have gone on to infect others. Similarly, Park and colleagues found that ignoring right truncation for a fast-growing epidemic with relatively long delays could result in underestimation of the mean delay distribution by up to 50% [23].
Not adjusting for bias can also lead to incorrect estimates of other parameters of interest that rely on accurate estimates of delays, such as the reproduction number. The time-varying reproduction number, Rt, is the average number of secondary cases caused by a single infected individual in a population. Gostic and colleagues showed how mis-specifying the mean, variance, or form of the generation interval led to biased estimates of Rt when using 3 empirical methods on synthetic data [28]. The bias was greatest early in the epidemic. Underestimation of Rt during this phase could lead to an insufficient public health response. Similarly, if changes in the generation interval (or serial interval) over the course of the epidemic are not accounted for, Rt may not be accurate [29].
In the aftermath of the COVID-19 pandemic, there is considerable need for and interest in the estimation and reporting of epidemiological parameters, including delays, with at least 2 R packages [30,31], a World Health Organization working group [32], and meta-analyses of priority pathogens [33–35] working to collect and make these data accessible. In this perspective, we present best practices for the estimation and reporting of epidemiological delays, illustrated by examples for the incubation period and serial interval of directly transmitted (person-to-person) infectious diseases. Our recommendations are based on our experience estimating and using delays across multiple outbreaks and recent methodological work [23].
To make these best practices easier to follow, we developed 2 checklists (Tables 2 and 3) and a flowchart (Fig 3), which can be used to understand which biases need to be adjusted for based on available data. To provide context for our recommendations, we additionally provide details about the data needed to estimate delays and how they should be prepared for analysis. Then, we discuss biases that can affect the estimation of epidemiological delay distributions, followed by strategies that can be used to reduce the impact of these biases. Technical details about these biases and how to adjust for them can be found in Park and colleagues [23] along with extensive simulation and case studies. Our paper goes beyond the work of Park and colleagues [23] by offering practical guidelines, a suggested workflow, and checklists, for a broader technical audience, such as modelers at public health agencies.
If you have an estimate of the backwards distribution from the literature, see the section on “Other considerations” for advice.
Examples of diseases for each checklist item were selected based on convenience (either papers written by this study’s authors or those encountered during the course of their work).
Data to estimate delays
Primary and secondary events can be observed (e.g., symptom onset time) or unobserved (e.g., usually infection time). Data to estimate the incubation period include the times of probable exposure (from which infection time can be inferred) and perceived symptom onset for each case. Data to estimate the serial interval are those in which symptom onset has been observed for primary and secondary cases. Examples of study designs or public health activities that generate such data include contact tracing [24,36], prospective cohort studies [37–39], household studies [40], or other types of intensive cohort monitoring [41]. Data from passive surveillance, which involves healthcare providers reporting cases to public health agencies, can also be used to estimate delay distributions; however, key information about exposures (e.g., dates, settings, and types of contact) may be missing, incomplete, or abstracted from other variables [42,43].
Biases in delay data
Three main biases can affect the estimation of epidemiological delay distributions are the following: (1) censoring; (2) right truncation bias; and (3) dynamical (or epidemic-phase) bias (Table 1). All of them affect both single-individual delays, such as the incubation period, and pair-dependent delays, such as the serial interval.
Censoring is knowing that an event occurred but not precisely when. Data can be right censored (the event is known to have occurred after a certain time), left censored (the event is known to have occurred before a certain time), or interval censored (the event is known to have occurred within a certain time interval). In epidemiological delay data, censoring can affect either primary or secondary events (single interval censoring) or both (double interval censoring) [44]. Epidemiological data are almost always doubly interval-censored due to discretization of the end points of the intervals being measured. For example, when reporting occurs daily with a cutoff at midnight, a patient could experience the event of interest (e.g., symptom onset) at any time between 12:00 AM and 11:59 PM on a particular day. Double interval censoring is shown in Fig 1 by the brackets around each event (circles and squares). Events occur anywhere on the x axis, which represents continuous time, but the reporting of events only occurs at discrete time points, or observation times. Some events are prone to longer censoring intervals than others (e.g., exposure intervals may be longer than 1 day for cases with multiple possible exposures). Not or incorrectly accounting for censoring of event intervals can lead to biased estimates of a delay [23].
Right truncation is defined as the inability to observe intervals (e.g., incubation periods) greater than a threshold (e.g., greater than the number of days elapsed since infection). It typically applies to real-time settings, when events with longer intervals may not have occurred yet, leading to an overrepresentation of shorter intervals when estimating delays. Right truncation is common in data where case ascertainment depends on the secondary event, e.g., we rarely observe an individual’s incubation period until after symptoms develop. Not accounting for right truncation can lead to underestimating the mean delay [23]. Although right truncation is mainly a problem for real-time analyses, retrospective data can be right-truncated if surveillance ended prematurely.
Right truncation should not be confused with right censoring. The latter occurs when we observe the primary event of a case or future case but cannot observe it long enough to witness its secondary event [45], which could be due to, for instance, a study ending prematurely. As a result, we only know that the secondary event did not occur during the observation period and therefore have a right-censored interval for a data point. Right censoring is shown in Fig 1A. The intervals on the top and bottom of the panel are included in the analysis, but we do not know when the secondary events will occur because they happen after the observation time. In contrast, right truncation means that certain delays are completely missing from our data as observing primary events depends on identifying secondary events first. Right truncation is shown in Fig 1B. Here, the intervals at the top and bottom of the panel are not included in the analysis; we are unaware of these data points because their secondary events occur after the observation time.
Dynamical bias is another type of common sampling bias which is analogous to right truncation. During the increasing phase of an epidemic, patients with short delays are overrepresented in the recent data, leading to underestimation of delay intervals. Conversely, when the epidemic is decreasing, patients with long delays are overrepresented in the recent data, leading to the overestimation of delay intervals. Dynamical bias is especially problematic during periods of exponential growth and decay of cases when cases are exponentially more and less likely, respectively, to be infected recently rather than further back in time.
Measuring epidemiological delays
We aim to estimate the true underlying distribution for each epidemiological delay which characterizes the time between the primary and secondary event. In general, we assume that this distribution does not change over the course of an epidemic (although this may not always be the case [29]). Cases can enter a data set due to observation of either their primary or secondary event (shown by the arrows in Fig 2). Regardless of how data were collected, we can organize our data into cohorts using either a forward or backward approach. For the forward approach, we start from primary events that occurred during the same period and prospectively determine when the secondary events occurred—the resulting distribution of the delays is the forward distribution (Fig 2A and 2C). In contrast, for the backward approach, we start with secondary events that occurred during the same period and retrospectively determine when the primary events occurred—the resulting distribution of the delays is the backward distribution (Fig 2B) [23].
Data observed in real-time can be subject to either right truncation or right censoring. Right truncation causes the observed forward distribution to be shorter than the true underlying distribution and has the largest effect when the epidemic is growing because relatively many recently infected individuals with long delays are not observed. In a declining epidemic, right truncation will have a smaller impact on the forward distribution because the proportion of recently infected individuals with unobserved secondary events is lower [23]. Excluding right-censored data from the analysis is equivalent to right truncating the data and leads to underestimation of the delay. Backward distributions are not susceptible to right truncation but can have a delay distribution that is shorter or longer than the forward distribution and the true underlying distribution depending on the phase of the epidemic (i.e., dynamical bias). Both right truncation and dynamical biases are minimal if data from the entire epidemic are available and included in a delay estimate [23].
Fig 1 shows the impact of different biases on the forward and backward distributions. We recommend always analyzing delay distributions as forward distributions and accounting for potential biases (i.e., censoring and right truncation) as this approach does not require additional information on incidence trajectories, which are used to adjust for dynamical bias in the backward distribution [23].
Adjusting for common biases
Fig 3 shows a decision tree for assessing which biases need to be addressed depending on the approaches taken for data collection and processing. For each of 3 possible scenarios, it includes an explanation about the potential impact of the biases as well as the methods needed to adjust for them.
In general, adjusting for double interval censoring involves estimating the conditional probabilities of the primary and secondary events occurring between their observed lower and upper bounds. Interval censoring should always be adjusted for, and the adjustment method is the same irrespective of the epidemic phase [23]. Right censoring can be handled using methods for survival analysis, such as the Kaplan–Meier approach [45]. Adjusting for right truncation involves normalizing the probability of observing a given delay from the untruncated forward distribution by the probability of observing any delay before the final observation time, and adjusting for dynamical bias involves incorporating the epidemic trajectory (e.g., the growth or decay rate of the epidemic) into the analysis [23,26,46,47].
There are several available methods and tools for estimating epidemiological delay distributions; however, most of these approaches have not been validated or do not correct for all potential biases [23]. One example is the coarseDataTools R package developed by Reich and colleagues [48]. This tool has been validated and can correct for double interval censoring, but it does not adjust for right truncation or dynamical bias. In contrast, epidist [49], an R package developed by some of the authors of this study, contains methods which can adjust for all 3 potential biases. In Park and colleagues [23], a simulation study was used to evaluate multiple methods and found that the approximate latent variable censoring and truncation method emerged as the best performer for both real-time and retrospective analyses for most real-world use cases. This method corresponds to the double interval censoring and right truncation adjusted model developed by Ward and colleagues [24] which we recommend.
Ward and colleagues’ approach estimates the probability of observing a secondary event conditional on observing the primary event by a given final observation time [24]. Estimated event times for each case are included in the model as unobserved, or latent, variables, and uniform prior distributions are used for both the primary and secondary event times, which can accommodate censoring intervals of arbitrary length. However, when censoring intervals are long, the event time distribution within the censoring interval will deviate from the uniform approximation (as its shape depends on underlying epidemic dynamics) and should be taken into account [24]. This model has important limitations. For example, Park and colleagues [23] found that this method was not able to estimate the mean or standard deviation as well in epidemic simulations characterized by very rapid exponential growth and long delays because primary event times are poorly approximated by a uniform distribution.
Certain practices should be avoided if using an alternative approach to adjust for biases in delays. For example, we suggest avoiding approaches that adjust for right truncation and dynamical biases simultaneously because they lead to overestimation of the mean delay by overcompensating for intervals that have not yet been observed [23]. We also do not recommend using a midpoint imputation rule (i.e., use the midpoint of the interval as the “observed” value and construct downstream inferences based on that imputation) on interval-censored data as such imputation approaches may introduce bias [23,50].
Additional modeling recommendations
Beyond correctly adjusting for biases, there are several common issues with reported epidemiological delays that may lead to biased conclusions when used in practice or impact their ability to be used at all. Historically, the incubation period and serial interval were often reported using only the mean (and sometimes the range) [51–54]. However, models can be fitted to delay data to adjust for some of the biases we have described and better characterize the tail of the distribution. Assuming a modeling approach is taken, we summarize our recommendations for estimating and reporting for epidemiological delay distributions in Table 2 and give more details in the sections “Reporting epidemiological delay distributions” and “Reporting the incubation period and serial interval.” Table 2 contains details about each recommendation, examples of diseases for which it has been implemented, and possible solutions. It is divided into 2 sections, estimation and reporting. Whenever estimating delays, we recommend going through the table to make sure all the steps are taken.
Fit multiple probability distributions.
We recommend fitting multiple probability distributions to summarize the empirical delay distribution [55]. This approach has greater utility for users of the estimates compared to nonparametric approaches. Use appropriate model comparison criteria (e.g., widely applicable information criterion [WAIC] or leave-one-out information criterion [LOOIC] for Bayesian models) to suggest the best-fitting model alongside visual checks. Common distributions for epidemiological delays in the literature include the gamma, lognormal, and Weibull distributions [56]. For delays that can have negative values, distributions that accept negative values, such as the skew-normal or skew-logistic distributions [57], may be used, or less ideally, the delay data may be shifted to allow for fitting of distributions that only allow positive numbers [58]. Mixture distributions may be appropriate for some delays, such as those with bimodal distributions [59–62]. Fitting parametric distributions may not be appropriate for all delay distributions. Semi- or nonparametric approaches, such as hazard models, may be considered [63]. Nonparametric estimates can also be used to assess the relative fit of parametric models [27].
Visualize the distributions.
It is also important to visualize the fitted distributions to check that they fit the data [36]. When doing so, we recommend visualizing the estimated distribution in conjunction with the modeled observation process (e.g., double interval censoring and right truncation). In other words, estimate the latent (continuous) distribution. Then, from the latent distribution, simulate elements of the observation process, such as double interval censoring (e.g., for date-level censoring this will transform continuous delay times into an integer number of days elapsed) and right truncation (this will change the shape of the observed distribution, relative to the latent distribution) [49]. Not accounting for the observation process after estimating the latent distribution makes visual assessment of the fit difficult because the observation data and the latent distribution may differ in shape and data type. For example, Sender and colleagues illustrate how good visualizations can help make intuitive comparisons across different distributions [64].
Correctly convert parameters.
Care should be taken when converting the parameters of fitted probability distributions to the summary statistics of interest. For example, the gamma distribution may use either a scale or a rate parameter, in addition to its shape parameter [65], while the standard lognormal parameters, log mean and log sd do not correspond to the log of the mean and the log of the standard deviation of the lognormal [66]. Some R packages contain functions to perform parameter conversion, such as EpiNow2 (lognormal) [67], epitrix (gamma) [68], mixR (gamma, lognormal, and Weibull) [69], and epiparameter (gamma, lognormal, Weibull, negative binomial, and geometric) [30].
Add subgroups or stratify estimates.
If sample size allows, we recommend stratifying delay estimates whenever there are hypothesized differences across groups as delays, such as the incubation period and the serial interval, may vary by route of exposure [15], viral species [70] or clade, disease severity, sex [71], age [71], vaccination status, or other factors. Ideally, this stratification should be done jointly in a statistically robust framework [49,63,72]. Wider application of joint modeling approaches could be achieved with more availability of easy-to-use tools [49].
Check model diagnostics.
When Bayesian methods are used, visualize the posterior distribution against data and check model diagnostics, such as trace plots, the potential scale reduction statistic (R-hat), divergent transitions, and effective sample sizes, and report them [73–75]. Convergence issues may indicate that the model is mis-specified, making the results unreliable. For more advice about using Bayesian methods, see [73,74,76].
Reporting epidemiological delay distributions
We recommend reporting an estimate of variability (e.g., standard deviation or dispersion) along with central tendency (e.g., mean or median) for all estimated delay distributions (sometimes more than one distribution fits the data similarly well as in [36]) alongside the quantiles and underlying parameters of the fitted distributions. These quantities are often used as inputs in infectious disease models and can inform both clinical practice and public health policy [8]. If possible, the probability density function should be specified to avoid ambiguity about the parameters. For Bayesian analyses, samples of the posterior distribution should be made available as summarizing estimates may obfuscate valuable information about the correlations between parameters and the shape of the posterior distributions.
All summary statistics should always be accompanied by credible intervals or confidence intervals for Bayesian and frequentist analyses, respectively (usually 90% or 95% with the width of the reported interval also being reported). High uncertainty in parameter estimates can have substantial impacts on downstream modeling [77,78] and can indicate that more data need to be collected.
Estimates of delays should be accompanied by contextual information to aid in interpretation. For example, we recommend reporting the study sample size; the epidemic curve; which, if any, control measures are in place; and summary statistics on age, sex, geographic location, vaccination status, and possible exposure route(s). Control measures and summary statistics can be used to assess generalizability of the estimates (see earlier advice on stratification of estimates). The epidemic curve can indicate at which stage of the epidemic the analysis took place and whether the outbreak is now over (i.e., whether certain biases need to be adjusted for).
Code and data should be uploaded to repositories, such as GitHub (https://github.com/) or Zenodo (https://zenodo.org/), to ensure reproducibility of the analysis and facilitate re-use of the code. Apart from allowing others to reproduce, validate and potentially improve analyses, providing data along with estimates of delay distributions also ensures that the estimates can be integrated in future pooled estimation efforts as methods continue to be improved. These data should ideally be provided in linelist format with all necessary information required for estimation (e.g., the left and right boundaries of the possible infection and symptom onset times for the incubation period [44]). Importantly, the data should be anonymized/de-identified to protect patient privacy according to local health data laws and regulations. If data cannot be shared, we recommend at minimum providing samples of the posterior distribution in a permanent online repository to facilitate future re-analyses (as in [24]).
Reporting the incubation period and serial interval
In addition to the checklist for reporting epidemiological delay distributions, we recommend additional considerations specific to the incubation period and serial interval in Table 3. Like Table 2, this checklist contains details about each recommendation, examples from real outbreaks, and possible solutions. It should be used each time an incubation period or serial interval is estimated.
For the incubation period, a case may have had multiple possible exposures prior to symptom onset, especially when community transmission of a pathogen is high. If a case reports multiple exposures, we recommend defining an exposure window that includes all possible exposure dates [7], using disjointed exposure windows where appropriate. Other methods that take this uncertainty into account could be used (such as [79] who used a Bayesian framework to infer the incubation period of the infector). We caution against restricting the analysis to cases with a high degree of certainty about their exposure periods as this can introduce biases [80].
For the serial interval, we only use case pairs where we are fairly confident transmission has occurred [58] (usually based on exposure information collected from patient interviews, see [36,81]). Although this approach could bias the serial interval towards specific lengths of intervals, it is usually preferable than to using mis-specified case pairs. Bias from including mis-specified case pairs in the analysis is likely larger than that from removing pairs between which transmission likely did not occur; however, future research should formally assess this convention with simulations. In terms of the direction of transmission, a number of approaches can be taken to order the case pairs. Where there is strong evidence that presymptomatic transmission does not occur for the disease of interest, reported negative intervals are presumed to be erroneous, and can be removed [36] or reversed [24]. Some studies assume the direction of transmission between epidemiologically linked cases based on the date order of symptom onset [24]; where negative serial intervals are possible, this should be avoided. It is also possible to use genomic data to order the pairs [82,83]. The ideal approach where negative intervals are possible is to use information about pair ordering and fit a distribution that allows for negative values (see “Additional modeling recommendations”). When no such information is available, a method that does not rely on knowing the order of case pairs would be ideal as this would enable the use of more available data and avoid biases from certain types or durations of exposure being easier or more difficult to link epidemiologically. Although such methods have been developed, they assume a fully sampled population [84,85]. It is important to be transparent about the approach taken. Uncertainty in the source of infection (such as the potential for multiple possible infectors) should also be considered.
Other considerations
How to best use new data.
Delays should be (re-)estimated when possible, especially if current estimates are poor or lacking. New estimates can be compared to those from previous epidemics or from a different phase of the same epidemic. One could consider any new data set on its own. Alternatively, using mixed effects models to partially pool information across different outbreaks is likely a better use of available data. When choosing an approach, it is important to consider whether the primary modes of transmission or pathogen properties are different than in the past [56,86].
Choice of prior distributions.
When using Bayesian methods to estimate epidemiological delay distributions, it is important to think about the choice of prior distributions. Some R packages that estimate epidemiological delays have default prior distributions [48,49]. Users of these tools should carefully consider the appropriateness of default prior distributions for their analyses and should explore the impact of different prior distributions on their results. We recommend against using uninformative prior distributions, especially uniform prior distributions [87], for the parameters of epidemiological delay distributions. When the delay distribution is already well reported in the literature, this knowledge can be used to inform prior distributions; however, the methods should be clearly communicated and accompanied by estimates generated from weakly informative prior distributions as sensitivity analyses.
Meta-analyses.
To reduce uncertainty from small sample sizes, some researchers have combined estimates of epidemiological delays from different studies through meta-analysis [34,35] or pooled analysis (re-analyzing published individual-level data) [8,88]. The latter method is preferred but may not always be possible. If performing a meta-analysis, we recommend performing sensitivity analyses when some estimates from the literature have not been corrected for bias [89]. Published estimates can also be adjusted for bias post hoc by using the relationship between the backward and forward distributions, as in Park and colleagues [89]. Some authors have designed custom quality assessment scales to assess bias in studies included in meta-analyses of epidemiological parameters [33,34].
Time-varying delays.
While some delay distributions are expected to remain stationary during an epidemic wave, others can change over time in response to interventions [29] and changes in reporting, among other factors. For example, Ali and colleagues found that the serial interval of COVID-19 in mainland China shortened after non-pharmaceutical interventions were implemented in early 2020 [29]. We can study time-varying delays by analyzing changes in forward delay distributions across cohorts. When doing so, we still need to account for both censoring and right truncation. We do not recommend inferring time-varying delays from backward delay distributions because dynamical bias also causes changes in this distribution even when the forward distribution remains stationary.
Discussion
Epidemiological delay distributions are key parameters for preparedness and response to epidemics and pandemics. Their importance has been highlighted during the COVID-19 pandemic [29], the global mpox (formerly known as monkeypox) outbreak in 2022 [7], and other epidemics over the last 2 decades [13]. We have focused on the incubation period and serial interval, for which estimates are often made at the beginning of an epidemic when contact tracing data are available to support early characterization of the pathogen. While the estimates are most useful for real-time response during this time, they are also the most susceptible to bias. Indeed, adjusting for bias when estimating delay distributions is one of the most important recommendations we highlight as not doing so may lead to the propagation of bias into downstream modeling [23] and therefore an incorrect understanding of an epidemic (e.g., over- or underestimating the risk to the host population) [25]. In addition to adjusting for bias, estimates need to be clearly and fully reported to maximize utility and make the most of data that are both costly and difficult to collect [90,91]. Our recommendations can assist with this.
A limitation of current methods for correcting common biases is that they do not fully account for time-varying changes in delay distributions [23]. Future work on delay distributions or nowcasting (which demonstrates how to model time-varying delays using time-to-event modeling) [92–94] should extend current methods or develop new methods to account for these changes.
Many of the best practices outlined in this paper also apply to other epidemiological delays. However, there are issues which we did not cover. For example, we did not focus on methods for estimating delay distributions for vector-borne diseases, such as dengue and Yellow fever, as these require additional considerations (e.g., accounting for vector biology) [13,95]. Also, we did not aim to provide a systematic review; rather, we provided insights based on our experiences. The examples presented in this work were selected to illustrate specific points.
In conclusion, we have provided recommendations in the form of 2 checklists for generating and evaluating epidemiological delay distributions. We gave examples of good practice for the incubation period and serial interval from various infectious disease outbreaks over the last few decades, though few examples in the literature incorporate all the best practices outlined in this paper. We hope that our recommendations will provide clarity and structured guidance about what should be reported and how to adjust for biases in delay data. We also hope our flowchart and checklists will be adopted by the infectious disease modeling community to understand the limitations of existing estimates and improve future estimates.
Disclaimer
The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the CDC, US Department of Health and Human Services, NIHR, UK Health Security Agency, or the UK Department of Health and Social Care.
References
- 1. Home | NSF NEON | Open Data to Understand our Ecosystems. [cited 2024 Aug 23]. Available from: https://www.neonscience.org/.
- 2.
EarthRanger: Protecting Wildlife With Real-Time Data. [cited 2024 Aug 23]. Available from: https://www.earthranger.com/.
- 3. Paz A, Silva TS, Carnaval AC. A framework for near-real time monitoring of diversity patterns based on indirect remote sensing, with an application in the Brazilian Atlantic rainforest. PeerJ. 2022;10:e13534. pmid:35789655
- 4. Panigrahi S, Maski P, Thondiyath A. Real-time biodiversity analysis using deep-learning algorithms on mobile robotic platforms. PeerJ Comput Sci. 2023;9:e1502. pmid:37705641
- 5. Venverloo T, Duarte F. Towards real-time monitoring of insect species populations. Sci Rep. 2024;14:18727. pmid:39134595
- 6. Cornford R, Spooner F, McRae L, Purvis A, Freeman R. Ongoing over-exploitation and delayed responses to environmental change highlight the urgency for action to promote vertebrate recoveries by 2030. Proc Biol Sci. 2023;290:20230464. pmid:37072041
- 7. Charniga K, Masters NB, Slayton RB, Gosdin L, Minhaj FS, Philpott D, et al. Estimating the incubation period of monkeypox virus during the 2022 multi-national outbreak. medRxiv 2022; 2022.06.22.22276713.
- 8. Lessler J, Reich NG, Brookmeyer R, Perl TM, Nelson KE, Cummings DA. Incubation periods of acute respiratory viral infections: a systematic review. Lancet Infect Dis. 2009;9:291–300. pmid:19393959
- 9. Overton CE, Abbott S, Christie R, Cumming F, Day J, Jones O, et al. Nowcasting the 2022 mpox outbreak in England. Lau EHY, editor. PLoS Comput Biol. 2023;19:e1011463. pmid:37721951
- 10. Lavezzo E, Franchin E, Ciavarella C, Cuomo-Dannenburg G, Barzon L, Del Vecchio C, et al. Suppression of a SARS-CoV-2 outbreak in the Italian municipality of Vo’. Nature. 2020;584:425–429. pmid:32604404
- 11. WHO Ebola Response Team. Ebola Virus Disease in West Africa—The First 9 Months of the Epidemic and Forward Projections. N Engl J Med. 2014;371:1481–1495. pmid:25244186
- 12. Fraser C, Donnelly CA, Cauchemez S, Hanage WP, Van Kerkhove MD, Hollingsworth TD, et al. Pandemic Potential of a Strain of Influenza A (H1N1): Early Findings. Science. 2009;324:1557–1561. pmid:19433588
- 13. Ferguson NM, Cucunubá ZM, Dorigatti I, Nedjati-Gilani GL, Donnelly CA, Basáñez M-G, et al. Countering the Zika epidemic in Latin America. Science. 2016;353:353–354. pmid:27417493
- 14. Ward T, Glaser A, Overton CE, Carpenter B, Gent N, Seale AC. Replacement dynamics and the pathogenesis of the Alpha, Delta and Omicron variants of SARS-CoV-2. Epidemiol Infect. 2023;151:e32. pmid:36535802
- 15. Reynolds MG, Yorita KL, Kuehnert MJ, Davidson WB, Huhn GD, Holman RC, et al. Clinical Manifestations of Human Monkeypox Influenced by Route of Infection. J Infect Dis. 2006;194:773–780. pmid:16941343
- 16. Galmiche S, Cortier T, Charmet T, Schaeffer L, Chény O, Von Platen C, et al. SARS-CoV-2 incubation period across variants of concern, individual factors, and circumstances of infection in France: a case series analysis from the ComCor study. Lancet Microbe. 2023;4:e409–e417. pmid:37084751
- 17. Cramer E, Ray E, Lopez V, Bracher J, Brennen A, Rivadeneira A, et al. Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the United States. Proc Natl Acad Sci U S A. 2022;119:e2113561119. pmid:35394862
- 18. Borchering RK, Healy JM, Cadwell BL, Johansson MA, Slayton RB, Wallace M, et al. Public health impact of the U.S. Scenario Modeling Hub. Epidemics. 2023;44:100705. pmid:37579585
- 19. Clay PA, Pollock ED, Saldarriaga EM, Pathela P, Macaraig M, Zucker JR, et al. Modeling the impact of prioritizing first or second vaccine doses during the 2022 mpox outbreak. Infect Dis (except HIV/AIDS). 2023 Oct.
- 20. Nishiura H, Linton NM, Akhmetzhanov AR. Serial interval of novel coronavirus (COVID-19) infections. Int J Infect Dis. 2020;93:284–286. pmid:32145466
- 21. Liu Y, Centre for Mathematical Modelling of Infectious Diseases nCoV Working Group, Funk S, Flasche S. The contribution of pre-symptomatic infection to the transmission dynamics of COVID-2019 [version 1; peer review: 3 approved]. Wellcome Open Res. 2020;5:58. pmid:32685697
- 22. Fraser C, Riley S, Anderson RM, Ferguson NM. Factors that make an infectious disease outbreak controllable. Proc Natl Acad Sci U S A. 2004;101:6146–6151. pmid:15071187
- 23. Park SW, Akhmetzhanov AR, Charniga K, Cori A, Davies NG, Dushoff J, et al. Estimating epidemiological delay distributions for infectious diseases. medRxiv. 2024.
- 24. Ward T, Christie R, Paton RS, Cumming F, Overton CE. Transmission dynamics of monkeypox in the United Kingdom: contact tracing study. BMJ. 2022:e073153. pmid:36323407
- 25. Overton CE, Stage HB, Ahmad S, Curran-Sebastian J, Dark P, Das R, et al. Using statistics and mathematical modelling to understand infectious disease outbreaks: COVID-19 as an example. Infect Dis Model. 2020;5:409–441. pmid:32691015
- 26. Britton T, Scalia TG. Estimation in emerging epidemics: biases and remedies. J R Soc Interface. 2019;16:20180670. pmid:30958162
- 27. Seaman SR, Presanis A, Jackson C. Estimating a time-to-event distribution from right-truncated data in an epidemic: A review of methods. Stat Methods Med Res. 2022;31:1641–1655. pmid:34931911
- 28. Gostic KM, McGough L, Baskerville EB, Abbott S, Joshi K, Tedijanto C, et al. Practical considerations for measuring the effective reproductive number, Rt. Pitzer VEeditor. PLoS Comput Biol. 2020;16:e1008409. pmid:33301457
- 29. Ali ST, Wang L, Lau EHY, Xu X-K, Du Z, Wu Y, et al. Serial interval of SARS-CoV-2 was shortened over time by nonpharmaceutical interventions. Science. 2020;369:1106–1109. pmid:32694200
- 30. Lambert JW, Kucharski A. epiparameter. 2023. Available from: https://github.com/epiverse-trace/epiparameter.
- 31. Nash R, Morgenstern C, Cuomo-Dannenburg G, McCabe R, McCain K. epireview. [cited 2023 Dec 28]. Available from: https://mrc-ide.github.io/epireview/.
- 32.
WorldHealthOrganization/collaboratory-epiparameter-community. World Health Organization; 2023. Available from: https://github.com/WorldHealthOrganization/collaboratory-epiparameter-community.
- 33. McAloon C, Collins Á, Hunt K, Barber A, Byrne AW, Butler F, et al. Incubation period of COVID-19: a rapid systematic review and meta-analysis of observational research. BMJ Open. 2020;10:e039652. pmid:32801208
- 34. Cuomo-Dannenburg G, McCain K, McCabe R, Unwin HJT, Doohan P, Nash RK, et al. Marburg virus disease outbreaks, mathematical models, and disease parameters: a systematic review. Lancet Infect Dis. 2023:S1473309923005157. pmid:38040006
- 35. Doohan P, Jorgensen D, Naidoo TM, McCain K, Hicks JT, McCabe R, et al. Lassa fever outbreaks, mathematical models, and disease parameters: a systematic review and meta-analysis. medRxiv. 2024:p. 2024.03.23.24304596.
- 36. Madewell ZJ, Charniga K, Masters NB, Asher J, Fahrenwald L, Still W, et al. Serial Interval and Incubation Period Estimates of Monkeypox Virus Infection in 12 Jurisdictions, United States, May–August 2022. Emerg Infect Dis. 2023;29:818–821. pmid:36863012
- 37. Humphrey JH, Marinda E, Mutasa K, Moulton LH, Iliff PJ, Ntozini R, et al. Mother to child transmission of HIV among Zimbabwean women who seroconverted postnatally: prospective cohort study. BMJ. 2010;341:c6580–c6580. pmid:21177735
- 38. Antonucci G, Girardi E, Raviglione M, Ippolito G, Almi P, Angarano G, et al. Risk Factors for Tuberculosis in HIV-lnfected Persons: A Prospective Cohort Study. JAMA. 1995;274:143–148. pmid:7596002
- 39. Munywoki PK, Koech DC, Agoti CN, Lewa C, Cane PA, Medley GF, et al. The Source of Respiratory Syncytial Virus Infection In Infants: A Household Cohort Study In Rural Kenya. J Infect Dis. 2014;209:1685–1692. pmid:24367040
- 40. Levy JW, Cowling BJ, Simmerman JM, Olsen SJ, Fang VJ, Suntarattiwong P, et al. The Serial Intervals of Seasonal and Pandemic Influenza Viruses in Households in Bangkok, Thailand. Am J Epidemiol. 2013;177:1443–1451. pmid:23629874
- 41. Russell TW, Hellewell J, Jarvis CI, Van Zandvoort K, Abbott S, Ratnayake R, et al. Estimating the infection and case fatality ratio for coronavirus disease (COVID-19) using age-adjusted data from the outbreak on the Diamond Princess cruise ship, February 2020. Eurosurveillance. 2020:25. pmid:32234121
- 42.
Centers for Disease Control and Prevention. Introduction to Public Health. In: Public Health 101 Series. 2014 [cited 2024 Jan 9]. Available from: https://www.cdc.gov/training/publichealth101/surveillance.html.
- 43. Sharpe JD, Charniga K, Byrd KM, Stefanos R, Lewis L, Watson J, et al. Possible Exposures Among Mpox Patients Without Reported Male-to-Male Sexual Contact—Six U.S. Jurisdictions, November 1–December 14, 2022. MMWR Morb Mortal Wkly Rep. 2023;72:944–948. pmid:37651279
- 44. Reich NG, Lessler J, Cummings DAT, Brookmeyer R. Estimating incubation period distributions with coarse data. Stat Med. 2009;28:2769–2784. pmid:19598148
- 45. Dey T, Lipsitz SR, Cooper Z, Trinh Q-D, Krzywinski M, Altman N. Survival analysis—time-to-event data and censoring. Nat Methods. 2022;19:906–908. pmid:35927476
- 46. Park SW, Sun K, Champredon D, Li M, Bolker BM, Earn DJD, et al. Forward-looking serial intervals correctly link epidemic growth to reproduction numbers. Proc Natl Acad Sci U S A. 2021;118:e2011548118. pmid:33361331
- 47. Verity R, Okell LC, Dorigatti I, Winskill P, Whittaker C, Imai N, et al. Estimates of the severity of coronavirus disease 2019: a model-based analysis. Lancet Infect Dis. 2020;20:669–677. pmid:32240634
- 48. Reich NG, Lessler J, Azman A, Kamvar ZN. coarseDataTools: Analysis of Coarsely Observed Data. 2021. Available from: https://cran.r-project.org/web/packages/coarseDataTools/index.html.
- 49. Park SW, Abbott S. Estimating delay distributions accounting for common biases. 2023. Available from: https://github.com/parksw3/epidist-paper.
- 50. Leung K-M, Elashoff RM, Afifi AA. CENSORING ISSUES IN SURVIVAL ANALYSIS. Annu Rev Public Health. 1997;18:83–104. pmid:9143713
- 51. Nolen LD, Osadebe L, Katomba J, Likofata J, Mukadi D, Monroe B, et al. Extended Human-to-Human Transmission during a Monkeypox Outbreak in the Democratic Republic of the Congo. Emerg Infect Dis. 2016;22:1014–1021. pmid:27191380
- 52. Mertens PE, Patton R, Baum JJ, Monath TP. Clinical presentation of lassa fever cases during the hospital epidemic at Zorzor, Liberia, March-April 1972. Am J Trop Med Hyg. 1973;22:780–784. pmid:4745237
- 53. Martini G. Marburg virus disease. Postgrad Med J. 1973;49:542–546. pmid:4207635
- 54. WHO International Study Team. Ebola haemorrhagic fever in Sudan, 1976. Bull World Health Organ. 1978;56:247–270.
- 55. Nishiura H. Early efforts in modeling the incubation period of infectious diseases with an acute course of illness. Emerg Themes Epidemiol. 2007;4:2. pmid:17466070
- 56. Miura F, Van Ewijk CE, Backer JA, Xiridou M, Franz E, Op De Coul E, et al. Estimated incubation period for monkeypox cases confirmed in the Netherlands, May 2022 Eurosurveillance. 2022;27. pmid:35713026
- 57. Ferretti L, Ledda A, Wymant C, Zhao L, Ledda V, Abeler-Dörner L, et al. The timing of COVID-19 transmission. Epidemiology. 2020 Sep.
- 58. McAloon CG, Wall P, Griffin J, Casey M, Barber A, Codd M, et al. Estimation of the serial interval and proportion of pre-symptomatic transmission events of COVID− 19 in Ireland using contact tracing data. BMC Public Health. 2021;21:805. pmid:33906635
- 59. Akhmetzhanov AR, Cheng H-Y, Dushoff J. Incubation-period estimates of Omicron (BA.1) variant from Taiwan, December 2021–January 2022, and its comparison to other SARS-CoV-2 variants: a statistical modeling, systematic search and meta-analysis. Infect Dis (except HIV/AIDS); 2023 Jul.
- 60. Lover AA, Zhao X, Gao Z, Coker RJ, Cook AR. The distribution of incubation and relapse times in experimental human infections with the malaria parasite Plasmodium vivax. BMC Infect Dis. 2014;14:539. pmid:25280926
- 61. Klinkenberg D, Nishiura H. The correlation between infectivity and incubation period of measles, estimated from households with two cases. J Theor Biol. 2011;284:52–60. pmid:21704640
- 62. Hoffmann T, Alsing J. Faecal shedding models for SARS-CoV-2 RNA among hospitalised patients and implications for wastewater-based epidemiology. J R Stat Soc Ser C Appl Stat. 2023;72:330–345.
- 63. Abbott S, Lison A, Funk S, Choi-Hannah GH, Pearson CAB, et al. epinowcast/epinowcast: Epinowcast 0.2.2. Zenodo. 2023.
- 64. Sender R, Bar-On Y, Park SW, Noor E, Dushoff J, Milo R. The unmitigated profile of COVID-19 infectiousness. Elife. 2022;11:e79134. pmid:35913120
- 65.
Gamma distribution. Wikipedia. 2024. Available from: https://en.wikipedia.org/w/index.php?title=Gamma_distribution&oldid=1210997995.
- 66.
Log-normal distribution. Wikipedia. 2024. Available from: https://en.wikipedia.org/w/index.php?title=Log-normal_distribution&oldid=1210518171.
- 67. Abbott S, Hellewell J, Sherratt K, Gostic K, Hickson J, Badr HS, et al. EpiNow2: Estimate Real-Time Case Counts and Time-Varying Epidemiological Parameters. 2023. Available from: https://cran.r-project.org/web/packages/EpiNow2/index.html.
- 68. Jombart T, Cori A, Kamvar ZN, Schumacher D, Finger F, Whittaker C. epitrix: Small Helpers and Tricks for Epidemics Analysis. 2023. Available from: https://cran.r-project.org/web/packages/epitrix/index.html.
- 69. Yu Y. mixR: Finite Mixture Modeling for Raw and Binned Data. 2021. Available from: https://cran.r-project.org/web/packages/mixR/index.html.
- 70. Van Kerkhove MD, Bento AI, Mills HL, Ferguson NM, Donnelly CA. A review of epidemiological parameters from Ebola outbreaks to inform early public health decision-making. Sci Data. 2015;2:150019. pmid:26029377
- 71. Medley GF, Anderson RM, Cox DR, Billard L. Incubation period of AIDS in patients infected via blood transfusion. Nature. 1987;328:719–721. pmid:3614379
- 72. Lison A, Abbott S, Huisman J, Stadler T. Generative Bayesian modeling to nowcast the effective reproduction number from line list data with missing symptom onset dates. 2023 [cited 2024 Feb 21].
- 73. Hamra G, MacLehose R, Richardson D. Markov Chain Monte Carlo: an introduction for epidemiologists. Int J Epidemiol. 2013;42:627–634. pmid:23569196
- 74.
Lambert B. A student’s guide to Bayesian statistics. Los Angeles London New Delhi Singapore Washington DC Melbourne: SAGE; 2018.
- 75.
RStan: the R interface to Stan. [cited 2024 Apr 10]. Available from: https://mc-stan.org/rstan/articles/rstan.html.
- 76. Stan Development Team. Stan Modeling Language Users Guide and Reference Manual Version 2.34. 2022. Available from: https://mc-stan.org.
- 77. Elderd BD, Dukic VM, Dwyer G. Uncertainty in predictions of disease spread and public health responses to bioterrorism and emerging diseases. Proc Natl Acad Sci U S A. 2006;103:15693–15697. pmid:17030819
- 78. Pollett S, Johansson MA, Reich NG, Brett-Major D, Del Valle SY, Venkatramanan S, et al. Recommended reporting items for epidemic forecasting and prediction research: The EPIFORGE 2020 guidelines. PLoS Med. 2021;18:e1003793. pmid:34665805
- 79. Cowling BJ, Park M, Fang VJ, Wu P, Leung GM, Wu JT. Preliminary epidemiological assessment of MERS-CoV outbreak in South Korea, May to June 2015. Eurosurveillance. 2015:20. pmid:26132767
- 80. Cheng H-Y, Akhmetzhanov AR, Dushoff J. SARS-CoV-2 Incubation Period during Omicron BA.5–Dominant Period, Japan. Emerg Infect Dis. 2024:30. pmid:38146985
- 81. Miura F, Backer JA, Van Rijckevorsel G, Bavalia R, Raven S, Petrignani M, et al. Time Scales of Human Mpox Transmission in The Netherlands. J Infect Dis. 2023:jiad091. pmid:37014716
- 82. Campbell F, Didelot X, Fitzjohn R, Ferguson N, Cori A, Jombart T. outbreaker2: a modular platform for outbreak reconstruction. BMC Bioinformatics. 2018;19:363. pmid:30343663
- 83. Hall M, Woolhouse M, Rambaut A. Epidemic Reconstruction in a Phylogenetics Framework: Transmission Trees as Partitions of the Node Set. PLoS Comput Biol. 2015;11:e1004613. pmid:26717515
- 84. Forsberg White L, Pagano M. A likelihood-based method for real-time estimation of the serial interval and reproductive number of an epidemic. Stat Med. 2008;27:2999–3016. pmid:18058829
- 85. Wu KM, Riley S. Estimation of the Basic Reproductive Number and Mean Serial Interval of a Novel Pathogen in a Small Well-Observed Discrete Population. PLoS ONE. 2016;11:e0148061. pmid:26849644
- 86. Madewell ZJ, Yang Y, Longini IM, Halloran ME, Vespignani A, Dean NE. Rapid review and meta-analysis of serial intervals for SARS-CoV-2 Delta and Omicron variants. BMC Infect Dis. 2023;23:429. pmid:37365505
- 87. Stan Development Team. Prior Choice Recommendations. In: GitHub [Internet]. [cited 2024 Apr 16]. Available from: https://github.com/stan-dev/stan/wiki/Prior-Choice-Recommendations.
- 88. Rudolph KE, Lessler J, Moloney RM, Kmush B, Cummings DAT. Incubation Periods of Mosquito-Borne Viral Infections: A Systematic Review. Am J Trop Med Hyg. 2014;90:882–891. pmid:24639305
- 89. Park SW, Sun K, Abbott S, Sender R, Bar-on YM, Weitz JS, et al. Inferring the differences in incubation-period and generation-interval distributions of the Delta and Omicron variants of SARS-CoV-2. Proc Natl Acad Sci U S A. 2023;120:e2221887120. pmid:37216529
- 90. Fields VL, Kracalik IT, Carthel C, Lopez A, Schwartz A, Lewis NM, et al. Coronavirus Disease Contact Tracing Outcomes and Cost, Salt Lake County, Utah, USA, March–May 2020. Emerg Infect Dis. 2021;27:2999–3008. pmid:34698628
- 91. Zeng W, Samaha H, Yao M, Ahuka-Mundeke S, Wilkinson T, Jombart T, et al. The cost of public health interventions to respond to the 10th Ebola outbreak in the Democratic Republic of the Congo. BMJ Glob Health. 2023;8:e012660. pmid:37848269
- 92. Höhle M, An Der Heiden M. Bayesian nowcasting during the STEC O104:H4 outbreak in Germany, 2011. Biometrics. 2014;70:993–1002. pmid:24930473
- 93. Van De Kassteele J, Eilers PHC, Wallinga J. Nowcasting the Number of New Symptomatic Cases During Infectious Disease Outbreaks Using Constrained P-spline Smoothing. Epidemiology. 2019;30:737–745. pmid:31205290
- 94. Bastos LS, Economou T, Gomes MFC, Villela DAM, Coelho FC, Cruz OG, et al. A modelling approach for correcting reporting delays in disease surveillance data. Stat Med. 2019;38:4363–4377. pmid:31292995
- 95. Chan M, Johansson MA. The Incubation Periods of Dengue Viruses. Vasilakis N, editor. PLoS ONE. 2012;7:e50972. pmid:23226436
- 96. Lessler J, Ott CT, Carcelen AC, Konikoff JM, Williamson J, Bi Q, et al. Times to key events in Zika virus infection and implications for blood donation: a systematic review. Bull World Health Organ. 2016;94:841–849. pmid:27821887
- 97. Linton N, Kobayashi T, Yang Y, Hayashi K, Akhmetzhanov A, Jung S, et al. Incubation Period and Other Epidemiological Characteristics of 2019 Novel Coronavirus Infections with Right Truncation: A Statistical Analysis of Publicly Available Case Data. J Clin Med. 2020;9:538. pmid:32079150
- 98. Lauer SA, Grantz KH, Bi Q, Jones FK, Zheng Q, Meredith HR, et al. The Incubation Period of Coronavirus Disease 2019 (COVID-19) From Publicly Reported Confirmed Cases: Estimation and Application. Ann Intern Med. 2020;172:577–582. pmid:32150748
- 99. Plummer M, Best N, Cowles K, Vines K. CODA: Convergence Diagnosis and Output Analysis for MCMC. R News. (2006);6:7–11.
- 100. Gabry J, Mahr T. bayesplot: Plotting for Bayesian Models. Available: https://mc-stan.org/bayesplot/.
- 101. Guzzetta G, Mammone A, Ferraro F, Caraglia A, Rapiti A, Marziano V, et al. Early Estimates of Monkeypox Incubation Period, Generation Time, and Reproduction Number, Italy, May–June 2022. Emerg Infect Dis. 2022;28:2078–2081. pmid:35994726
- 102. Assiri A, McGeer A, Perl TM, Price CS, Al Rabeeah AA, Cummings DAT, et al. Hospital Outbreak of Middle East Respiratory Syndrome Coronavirus. N Engl J Med. 2013;369:407–416. pmid:23782161
- 103. McFarland SE, Marcus U, Hemmers L, Miura F, Iñigo Martínez J, Martínez FM, et al. Estimated incubation period distributions of mpox using cases from two international European festivals and outbreaks in a club in Berlin, May to June 2022. Eurosurveillance. 2023;28. pmid:37410383