2.1. Summary of Bayesian nonparametric N_e(t) inference

When analyzing pathogen evolution, we use an alignment of sampled pathogen genetic sequences as data. These sequences can either be collected at the same time, isochronous sampling, or at different points in time, heterochronous sampling. Here we are concerned with viruses that evolve rapidly with continuously collected samples so we will consider heterochronous sampling of DNA or RNA sequences aligned and stored in matrix , , , where n is the number of sequences and L is the alignment length. The sequences, , all ultimately share a common ancestry, and the evolution of the sequences from their most recent common ancestor is described by a bifurcating tree called a genealogy, denoted as .

We assume that given the genealogy, alignment sites are independent and identically distributed. The evolutionary changes in the nucleotides present at each alignment site, column of matrix , are modeled by a continuous-time Markov chain substitution model parameterized by vector . From a given viral genealogy and substitution rate matrix, the probability of observing sequences , can be calculated using an efficient dynamic programming algorithm [20]. Equipped with a model for the alignment, a model is needed for the pathogen’s genealogy.

The lower half of Fig 1 displays a genealogy relating five sequences, black tree tips, collected across four sampling times. Note the purple tips denote samples collected but not yet reported and available for use by the time of analysis. Sampling times are denoted by and sample sizes by with . In this set up we imagine generating the genealogy backwards in time starting from the most recent sampling time, s_m = 0. The branches of this evolutionary tree end at the sampling times , and the convergence, or coalescence, of two branches corresponds to a common ancestor of the two sequences. The tree’s branches coalesce until the most recent common ancestor of all of the samples, the root of the tree. The times of the coalescent events are denoted , with , and the most recent sampling time s_m = 0 will be denoted by t_n because this notation will make it convenient to define the joint density of coalescent times later.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Example of an effective population size trajectory (top figure) and corresponding genealogy (bottom figure), with reporting probabilities and corresponding unobserved tips denoted by purple coloring.

For a real-time analysis the reporting probability decreases as the collection date gets closer to present time, time zero.

https://doi.org/10.1371/journal.pcbi.1012970.g001

The effective population size, denoted N_e(t), is a time-varying measure of genetic diversity. The number of active lineages at a time t is the difference between the number of sampling and coalescent events between times 0 and t. The intervals I_i,k are defined by the sampling and coalescent times, so the number of active lineages, denoted l_i,k, in an interval is constant. For , the intervals that end in a coalescent event are denoted , for , and intervals that end with a sampling event are denoted , for with i>0. For a more detailed explanation of the I_i,k intervals, and a clarifying visual, see Lan et al. (2015) [21].

Coalescent models are continuous-time Markov chains used to model a genealogy from a sample of sequences [22]. Rodrigo et al. (1999) extended coalescent theory for heterochronous sampling to calculate the joint distribution of a genealogy given its sampling times, number of samples collected at each time, and the effective population size, as the product of conditional densities and tail probabilities of coalescent times [23]:

(1)

with the coalescent factors .

Assuming the effective population size trajectory N_e(t) is an unknown function in continuous time, the integral in Eq 1 is intractable. We adopt a common approach [24–26], well described by Lan et al. (2015), that discretizes the effective population size to be piecewise constant, with changes occurring at evenly spaced time points that form a regular grid , spanning from the most recent sampling time, , to the first coalescent time, [21]. In this approach we define , and approximate N_e(t) by . The ’s a priori follow a first order random walk: with . We adopt the common approach of using a gamma prior distribution for the hyperparameter .

With heterochronous sampling, it is likely that the frequency of sampling is related to the number of infections in the population (e.g., increased sampling intensity when there is an increase in infections). Additional factors may also influence the sampling intensity, such as time variable cost of sequencing a pathogen genome. We denote these additional factors as . In the preferential sampling model, we model sampling events as a Poisson Process with intensity that depends on such time-varying factors:

(2)

Note the sampling intensity can include interactions between the covariates and the log effective population size, but we do not include them in our model here. The coefficients ’s are assigned independent normal priors with means and variances . Since the effective population size is piecewise constant on the regular grid , for simplicity we require time-varying covariates also be piecewise constant on the same grid.

Altogether, the posterior we are interested in is

(3)

Approximation of this posterior via Markov Chain Monte Carlo (MCMC) is implemented in the phylodynamic software BEAST [18, 27]. This Bayesian inference is time and memory intensive though, so it is common in practice to estimate the genealogy first and assume the genealogy is known. When the genealogy is known the posterior of interest reduces to

(4)

Approximations of this posterior via MCMC and via Integrated Nested Laplace Approximations (INLA) [24] are implemented in the phylodynamic R package phylodyn [28].

2.2. Accounting for reporting delays

The time delay between collecting a sample and depositing that sample’s sequence into a database arose as a problem during the SARS-CoV-2 pandemic, because of the urgent need for up-to-date understanding of disease dynamics. Missing the most recent data is especially problematic for the preferential sampling model because of the dependency between the sampling intensity and the effective population size. Intuitively, a model that takes into account preferential sampling would underestimate the effective population size close to the present time due to the lack of observed samples. One possible solution of this problem is to use a coalescent model without the preferential sampling component, avoiding the dependency between the sampling intensity and the effective population size. While the biases from the missing data would be avoided with this strategy, unaccounted preferential sampling can result in biases, and wider credible intervals than those modeled with preferential sampling [17].

Another way to circumvent this missing data issue is to only use data up to a time when all of the data is likely to have been reported (e.g., data up to two months prior to time of analysis). For example, phylodynamics was used to compare SARS-CoV-2 lineages in England with data truncated by two weeks to avoid reporting delays in 2021 [6]. The major pitfall of this truncation strategy is the inability to perform real-time phylodynamics to inform outbreak mitigation, a problem that increases for locations or time periods with extensive reporting delays.

2.3. Incorporating reporting delay distribution into preferential sampling model

To mitigate effects of reporting delays on real-time phylodynamic analyses with preferential sampling, we propose incorporating information about the distribution of recent delays in the sampling intensity model. In the preferential sampling model sampling times are modeled as a Poisson process with intensity . Let be the probability that a sample collected at time t was sequenced and reported by the time of the analysis. Define the observed sampling times, to be the subset of the true sampling times, , that are reported by the time of analysis. Then the observed sampling intensity, , could be expressed as the product of the true sampling intensity and the probability of a sample being reported, resulting in a thinned Poisson process with intensity . Plugging Eq 2 into the definition of , we get the following new model for the log-sampling intensity

(5)

We refer to this proposed adjustment to the preferential sampling model as the delay-aware BNPR PS model.

2.4. Implementation of delay-aware BNPR PS model

Our proposed model assumes that the reporting probabilities are known, but in reality they would likely not be known, and they could be changing over time. Reporting and sampling dates are readily available metadata for pathogen sequences, so the user of our method is advised to use these reported metadata for a pathogen of interest to calculate the empirical cumulative distribution function (cdf) of the delays to approximate the reporting probabilities . Similarly to the effective population size and any covariates, the reporting probabilities must also be defined as piecewise constant across the regular grid . When approximating the reporting probabilities there are several considerations: Is there recent data that can provide information about reporting delay behavior? Should the reporting delays of recently reported sequence or recently sampled sequences be used? What should be considered recent?

Our proposed method is not applicable to situations without some data about recent reporting delays, such as the first few weeks of an outbreak where sequencing efforts are just beginning. In a situation with available data on recent reporting delays, it is important to recognize that the empirical cdf of sequences sampled during a predefined time window will provide biased estimates of reporting probabilities due to unreported sequences being censored. In the scenario with all sampling times known, regardless of sequences being reported by the time of analysis, survival analysis methods (e.g., Kaplan-Meier estimator) could be used to approximate the reporting probabilities. Otherwise, the empirical cdf of samples reported during a predefined time window should be used to estimate reporting probabilities.

When considering what time frame of reported sequences to use, it is important to look at the most recent reporting delays and examine if the delay behavior is changing overtime. The time frame of reported sequences to use should be chosen from the time of analysis back to a time where the reporting delay behavior continues to be relatively consistent. This is important because more data can provide better estimates, but reporting delays from sequences collected during a time period with very different delay behavior will not approximate current delay behavior well. Further investigation of these modeling decisions and their impact on the inference with the delay aware BNPR PS model is provided in S1 Appendix Sect 3.

We developed a new version of the R package phylodyn [28], phylodyn2 (https://github.com/CatalinaMedina/phylodyn2), which has a well-documented subset of the functionality of phylodyn, with the additional ability to account for reporting delays in real-time analyses through our proposed delay-aware BNPR PS model. The R package phylodyn included several posterior sampling strategies. For phylodyn2 we chose to focus on the Integrated Nested Laplace Approximations (INLA) based strategy to approximate the marginal posterior distributions because it is faster than the MCMC-based method. This INLA implementation formulates the model for sampling times as a Poisson regression [24]. A helpful observation is that the term in Eq 5 could be operationally viewed as an offset to this Poisson regression. However, the original phylodyn implementation did not allow for inclusion of a user specified offset term. That is one major change present in phylodyn2 that allowed for the implementation of the delay-aware BNPR PS model, where the reporting probabilities are calculated from a user specified vector of recent reporting delays.

Another way of viewing the term within the framework developed in [18] is considering it to be a time-varying covariate of the sampling intensity, with a coefficient of the value one. The appeal of this perspective is the ease of implementation with existing phylodynamic tools that allow for time-varying covariates in the sampling intensity, such as BEAST and phylodyn. One could specify as a regression covariate with a narrow prior for the coefficient of this term centered at one. This adds unnecessary randomness, since the coefficient of this term is theoretically one, but the ease of use makes this option worth exploring. This implementation is also available in phylodyn2, and its performance is examined in S1 Appendix Sect 1.1.

All code to reproduce the results in this paper can be found at https://github.com/CatalinaMedina/reporting-delays-in-phylodynamics-paper.

3.1. Simulations

We performed simulation studies to mimic real-time phylodynamic analyses in the presence of preferential sampling, aiming at two primary objectives. Firstly, to investigate the effects of reporting delays with currently available phylodynamic inferential strategies. Secondly, to compare the performance of our proposed model against the currently available strategies. Of key interest is how well the effective population size trajectory can be inferred close to the most recent sampling time.

Three real-time inferential strategies were considered for comparison: avoid modeling the sampling time dependency by using the the Bayesian nonparametric phylodynamic reconstruction (BNPR) model, model the sampling time dependency with the Bayesian nonparametric phylodynamic reconstruction with preferential sampling (BNPR PS) model, and model the sampling time dependency and reporting delays with our proposed delay-aware BNPR PS model. We also fit the BNPR PS model to all of the data, regardless of whether it was reported, to provide a retrospective baseline for the performance of these real-time inferential strategies.

We used three simulation scenarios with the same effective population size trajectory, but across different time periods so that the effects of reporting delays with different trajectory behavior near time zero could be investigated. The upper-left panel of Fig 2 shows the effective population size trajectory, as well as the most recent sampling time, the time of analysis, for each scenario. Since time is viewed in reverse, the most recent sample in simulation scenario C is time zero, and the earliest sample was 300 days prior. Scenarios A and B had sampling time periods of 150 days and 220 days, respectively. Scenario A is meant to resemble an initial outbreak, which would have fewest samples due to reporting delays. Scenario B allows us to examine behavior when there is a slight increase occurring near present time, but less with sampling. Lastly, in scenario C there is a decline near present time and the recent peak corresponds to more reported samples near time zero than in scenario B. In both scenarios B and C the recent change in trajectory direction would be expected to be difficult to detect due to reporting delays.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Simulation details: effective population trajectories (upper left plot), reporting probability by sampling time (upper right plot) obtained from the Washington state data, and histograms of sampling times from the last simulation of in each simulation scenario, approximately 1500 samples each, colored by whether sample was reported by time of analysis (bottom plots).

Each simulation scenario had a different time zero, i.e., time of latest sample (dashed lines). The earliest sampling time in each scenario was at the same point in the trajectory (dotted line).

https://doi.org/10.1371/journal.pcbi.1012970.g002

Sampling times were simulated from an inhomogeneous Poisson process with intensity . Coalescent times were simulated using the coalsim() function in the phylodyn R package [28], which uses a time-transformation technique where the coalescent likelihood is treated as an inhomogeneous Poisson process [29]. Parameter was set to 2 to create a reasonably strong preferential sampling effect and was selected to achieve a sample size of approximately 1500 samples, each with its own sampling time.

For each sampling time we simulated a random Bernoulli to indicate if a sample was reported by the time of analysis. Sampling times for each scenario are plotted in Fig 2 and colored by whether it was observed or not. To create realistic delays, the reporting probabilities were obtained from the empirical reporting delay distribution of SARS-CoV-2 sequences collected in the state of Washington. See the real data investigation results subsection for details, visualized in upper-right panel of Fig 2. The tips of the genealogy of the full tree that correspond to unreported samples were pruned from the tree, to get the observed genealogy. Each inference was performed with the INLA-based Bayesian phylodynamic inference implemented in the R package phylodyn2.

We will begin by discussing the results of a single simulation within a scenario, in order to better understand the patterns in the performance of each inference strategy across all of the simulations. Plots of the results of each inference strategy from a single simulation in each simulation scenario are available in S1 Appendix Sect 1.1. Fig 3 plots the true (solid lines) and inferred (dashed lines) effective population size trajectory for the 100 days prior to the most recently collected sample. Here we focus on the two options of real-time inference, the BNPR and BNPR PS models, and our proposed delay-aware BNPR PS model. While the ultimate goal is to be able to infer the true effective population size trajectory, it is useful to see how closely the data generating model can approximate the true trajectory, within simulation scenario C, where the trajectory of interest is on the decline at present time. This is why each plot also contains the BNPR PS inference performed retrospectively on all of the data, not just the observed data – this serves as a baseline to compare the inference of N_e(t) from each real-time inferential method. The white background indicates the time period of interest, where delays are probable, and conversely the gray background indicates the period where reporting delays are unlikely. We chose to use the 90th percentile of the Washington state data reporting delays distribution, which was 41 days in this case, as the cutoff for these two periods.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Comparison of real-time phylodynamic methods to infer the effective population size trajectory for a single simulation in scenario C, with reporting delays and preferential sampling present.

Median estimates of the effective population size and 95% credible intervals are plotted. The BNPR PS model was also applied retrospectively with all of the sampled sequences, regardless of if they were reported by time zero, to serve as a reference for comparison for the three real-time inference methods. The white background indicates the recent time period likely suffering from reporting delays, specifically where reporting probabilities (RPs) are below 90%, and is therefore the region of interest.

https://doi.org/10.1371/journal.pcbi.1012970.g003

In the first panel of Fig 3 we see real-time inference with the BNPR model, which ignores the dependency between N_e(t) and the sampling time. The BNPR model appears to have relatively low bias, but wide 95% credible intervals that increase in width near time zero. The real-time inference with the BNPR PS model stands out because of the bias which increases as time approaches the most recently collected sample. This demonstrates the bias introduced when using the data generating model, the BNPR PS model, when there are reporting delays present in the data. Alternatively, our implementation of the delay-aware BNPR PS model has less bias than the BNPR PS model near time zero and visibly narrower 95% credible intervals than the BNPR model near time zero.

The results identified from the single simulation in Fig 3 generally persist across all 500 simulations, in each of the three simulation scenarios, visualized in Fig 4. The plots present a seven-day moving average of the mean relative deviation, mean percent of 95% Bayesian credible intervals which covered the true value, and mean 95% credible interval width for each inference strategy in each simulation scenario. A moving average was chosen because a metric of the inference over the entire time period would be insufficient to describe how inference performance changes with proximity to time zero. Mean relative deviation is the most important of the three chosen metrics because it assess accuracy, of the point estimate, interval coverage was selected to examine the accuracy of the uncertainty of the estimates, and interval width was useful for assessing precision to compare those models with good accuracy and good coverage. Since the performance of these estimation strategies near time zero is of key interest, these plots were truncated to the most recent 100 days. To view the performance metrics results for all inferential methods considered see S1 Appendix Tables A-C.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Seven-day moving averages of the mean relative deviation, mean percent coverage, and mean interval width by the 95% Bayesian credible intervals, for each real-time phylodynamic strategies to infer the effective population size in each simulation scenario with preferential sampling (PS) and reporting delays in the observed data.

Real-time inference was performed with the Bayesian nonparametric phylodynamic reconstruction (BNPR) model, BNPR PS model, and with the delay-aware BNPR PS model with 500 simulations per scenario.

https://doi.org/10.1371/journal.pcbi.1012970.g004

Focusing on the time period of interest, the most recent 41 days, our proposed delay-aware BNPR PS model consistently has lower mean relative deviation than the BNPR method, in each seven-day moving average, though there is not much practical difference. The BNPR PS model has increasing relatively large mean relative deviation as sampling times decrease to time zero, in each simulation scenario. The absolute maximum mean relative deviations in scenario A are all achieved in the week prior to time zero are 0.15, 0.82, and 0.06 for the BNPR, BNPR PS, and our delay-aware BNPR PS model respectively. The 95% Bayesian credible intervals for the BNPR and our delay-aware BNPR PS model are consistently conservative, while the BNPR PS model’s 95% credible intervals’ coverage drops below 95% and approaches 0% as sampling time approaches time zero. Finally, while maintaining competitively low bias and high coverage, our proposed model consistently has lower mean 95% Bayesian credible interval widths than the BNPR model, with the difference between the two models increasing as sampling time approaches time zero.

3.2. Real data investigation: Washington state COVID dynamics

We used SARS-CoV-2 sequences from Washington state for the purpose of investigating the differences between a real-time phylodynamic analysis with and without our proposed method to account for reporting delays in genomic data. The SARS-CoV-2 sequences were accessed via the GISAID database available at https://gisaid.org/EPI_SET_220330me, for Washington state sampled between February 01, 2021 and August 01, 2021, inclusive [30]. This time period was of interest because researchers were regularly sequencing Washington samples at this point in the pandemic, and the reporting behavior is relatively consistent during this period. Fig 5 plots seven-day rolling average number of COVID-19 cases per 100,000 people in population in the state of Washington, the daily number of SARS-CoV-2 samples available in GISAID for Washington, colored by whether the sequence was sampled by August 01, 2021 (middle plot), and the empirical cumulative distribution function for sampling dates between July 01, 2021 and August 01, 2021.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Left plot is the seven-day rolling average of positive COVID-19 tests per 100,000 people in the state of Washington.

Middle panel shows number of SARS-CoV-2 genetic samples collected in Washington state, colored by whether the sample was reported by the time of analysis, August 1, 2021. Right panel shows empirical cumulative distribution of reporting delays from the month prior to time of analysis.

https://doi.org/10.1371/journal.pcbi.1012970.g005

The observed data are samples that had been sequenced and reported to GISAID on or before August 01, 2021, time zero of our analysis. The 90th percentile of the reporting delays distribution is 41 days. Since we are interested in the inference of N_e(t) when reporting delays are present, we chose to focus our attention on the most recent 41 days.

Genealogy estimation was performed in BEAST for each data set: all 500 sequences to represent a retrospective analysis, the observed 412 sequences to represent a real-time analysis, and 375 remaining sequences after truncating any sequence with sampling times larger than 41 days to avoid reporting delays in a near real-time analysis. We used the HKY substitution model with empirically estimated base frequencies [31], Bayesian Skygrid coalescent model [25, 32], and a Uniform prior on the clock rate between and [33]. The MCMC was run for iterations, logging parameters every 2000th iteration. The maximum clade credibility tree of the posteriors were used as the known genealogy in the phylodynamic reconstruction, for each of the three analyses. See S1 Appendix Sect 2 for more details about this analysis to obtain the genealogies.

Inference of the effective population size was performed with the same strategies used in our simulations to compare the performance of our proposed methods against available options. Fig 6 shows the inference of the effective population size for three modeling strategy: Bayesian nonparametric phylodynamic reconstruction (BNPR), BNPR with preferential sampling (BNPR PS), and our proposed delay-aware BNPR PS model.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Bayesian nonparametric phylodynamic reconstruction (BNPR) methods used to infer effective population size trajectory for SARS-CoV-19 in Washington state.

Each panel shows the inference from a real-time analysis on data suffering from reporting delays and from a retrospective analysis with completely reported data. The white background indicates the recent time period likely suffering from reporting delays.

https://doi.org/10.1371/journal.pcbi.1012970.g006

The retrospective analysis with all of the collected samples with the BNPR PS model infers a peak in transmission activity in mid April of 2021, dropping to a minimum in mid June, followed by a steady increase continuing into August 2021. The results for these analyses are consistent with the trajectory of COVID-19 cases for this time period, visualized in Fig 5, with approximately a two week delay which could be due to reporting delays in COVID test results.

When comparing the real-time analyses we see similar patterns as those identified in our simulations. Using the retrospective BNPR PS model for comparison we see near real time, the BNPR model suffers from low precision, the BNPR PS model’s credible intervals disagree with the retrospective analysis credible intervals, and our proposed delay-aware BNPR PS model is consistent with the retrospective BNPR PS model results, with higher precision than the BNPR model. This gain in precision found with our proposed model would have allowed real-time analysis to infer the increase near present time that the two currently available competitive methods underestimated.