Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Which COVID policies are most effective? A Bayesian analysis of COVID-19 by jurisdiction

Which COVID policies are most effective? A Bayesian analysis of COVID-19 by jurisdiction

  • Phebo D. Wibbens, 
  • Wesley Wu-Yi Koo, 
  • Anita M. McGahan
PLOS
x

Abstract

This paper reports the results of a Bayesian analysis on large-scale empirical data to assess the effectiveness of eleven types of COVID-control policies that have been implemented at various levels of intensity in 40 countries and U.S. states since the onset of the pandemic. The analysis estimates the marginal impact of each type and level of policy as implemented in concert with other policies. The purpose is to provide policymakers and the general public with an estimate of the relative effectiveness of various COVID-control strategies. We find that a set of widely implemented core policies reduces the spread of virus but not by enough to contain the pandemic except in a few highly compliant jurisdictions. The core policies include the cancellation of public events, restriction of gatherings to fewer than 100 people, recommendation to stay at home, recommended restrictions on internal movement, implementation of a partial international travel ban, and coordination of information campaigns. For the median jurisdiction, these policies reduce growth rate in new infections from an estimated 270% per week to approximately 49% per week, but this impact is insufficient to prevent eventual transmission throughout the population because containment occurs only when a jurisdiction reduces growth in COVID infection to below zero. Most jurisdictions must also implement additional policies, each of which has the potential to reduce weekly COVID growth rate by 10 percentage points or more. The slate of these additional high-impact policies includes targeted or full workplace closings for all but essential workers, stay-at-home requirements, and targeted school closures.

Introduction

Many countries, regions, and U.S. states (called ‘jurisdictions’ here) have implemented COVID-control policies such as stay-at-home requirements, workplace closings, school closings, restrictions on gatherings, international travel controls, testing and contact tracing. In this paper, we report on the results of a Bayesian analysis on which policies are most important to infection control. The model generates estimates of the marginal impact of each policy in a jurisdiction after accounting for (i) the overall portfolio of policies adopted by the jurisdiction, (ii) the levels at which the policies are implemented, (iii) the rigorousness of compliance within the jurisdiction, (iv) the jurisdiction’s COVID infections, COVID deaths, and excess deaths, and (v) the performance of the portfolio of policies in other jurisdictions. The purpose is to inform decisions about changing both the level and type of policies.

We find that, across the 40 jurisdictions, the relevance of each of the policies–and the levels at which they must be implemented to drive COVID infection growth to below zero–depends sensitively on compliance in the general population with a core group of policies that are relatively less constraining of social interaction than others. We call these socially tolerable policies the “core” group in our analysis. These include the cancellation of public events, the restriction of gatherings to fewer than 100 people, recommendation to stay at home, recommended restrictions on internal movement, implementation of a partial international travel ban, and coordinated public information campaigns. However, these relatively tolerable core policies are insufficient alone for COVID control in almost all jurisdictions. At least several of the higher-impact, harder-to-tolerate, more restrictive policies must be implemented to drive growth in COVID infections below zero. We refer to these additional policies, each of which must be implemented on top of the core policies to have the estimated effect, as the “additional high-impact” group. We find that, across the 40 jurisdictions, the portfolio of additional high-impact policies includes targeted or full workplace closings for all but essential workers, stay-at-home requirements, and targeted school closings. Each is estimated to reduce the virus growth rate by 10 percentage points or more. The optimal portfolio of additional high-impact policies appropriate for each jurisdiction depends sensitively on compliance with the core policies and on interactions between policies. All data, code, and modeling information used for this project are publicly available through links described below.

Assessing the consequences of changing policies depends on accurate information about infections and deaths [1,2]. Barriers to obtaining information about infections have been extensive, and include insufficient kit supply, test insensitivity, early underestimation of the importance of testing [3], and virus-shedding by asymptomatic patients who do not present themselves clinically [4]. Deaths from COVID have also been underreported as a result of non-diagnosis, non-treatment, and structural barriers. The model on which our assessments are based (described in the Model and Data section and the S1 File) involves Bayesian inference about historical levels of infection inferred from both COVID deaths and excess deaths occurring a few weeks subsequently. The model is tuned through simulations to achieve goodness-of-fit by jurisdiction. The model uses global information on case fatality rates [5] and infection-to-death lags that are nested probabilistically in models of transmission within jurisdictions. Data on weekly COVID infections and deaths by country and U.S. state is drawn from the Johns Hopkins University COVID dashboard. Data on weekly expected deaths is imported from The Economist dataset.

We incorporate the weekly levels reported in the Oxford COVID-19 Government Response Tracker (OxCGRT) database in eleven categories: School closing, workplace closing, cancel public events, restrictions on gatherings, close public transport, stay-at-home requirements, restrictions on internal movement, international travel controls, public information campaigns, testing, and contact tracing. The level of each policy is assessed daily on a scale that can include up to 4 stages. The lowest level of restriction is level 1 and the greatest level is level 4. The levels are customized for each policy. For example, level 1 of international travel controls is screening arrivals whereas level 4 is closing borders to international travel.

The first set of results is a report on the average impact on COVID infection of each of the eleven policies across all the represented jurisdictions in our model. The results depend on the simultaneous deployment of the various policies across jurisdictions. The second result is a report on the percentage of jurisdictions over time with a given policy in place. The third result is the current status of new infections and growth in infections by jurisdiction. Fourth, we estimate the weekly base growth rate in infections for each jurisdiction under the hypothetical scenario that each adopts four basic policies at level 1, i.e., school closing, workplace closing, cancel public events, and restrictions on gatherings. This allows us to estimate the base rate of compliance with the portfolio of policies by jurisdiction, which in turn enables estimation of the relative impact of each policy for relatively more vs. less compliant jurisdictions. Fifth, we estimate the extent to which a set of widely adopted core policies is effective in locally containing the pandemic, and which additional high-impact policies might potentially be needed to do so. The S1 File, which details the model, also reports the weekly new infections and deaths for each jurisdiction as well as additional sensitivity and robustness checks.

Existing research

Three streams of existing research are most relevant to our study: modeling studies that estimate and predict infection and death rates, medical studies that investigate the important drivers of disease spread, and studies that estimate the effectiveness of specific policies.

Epidemiological models

Many modeling approaches have been adopted to estimate infection rates, death rates, and disease spread. A well-established class of models relies on differential equations to analyze disease dynamics within susceptible, exposed, infected, and recovered (SEIR) “compartments” of populations [611]. The SEIR model and its variants (e.g., the SIR and SEIRS models) assume that the flow rates between compartments in the population are reflected in the history of the disease [9,10]. The SEIR framework has been used to study past COVID transmission and forecast future evolution [6,7,11,12]. Its variants have also produced insight on the importance of accounting for presymptomatic and asymptomatic individuals in order to curb COVID spread [13,14]. Challenges associated with SEIR-type models are that it is difficult to calibrate models (especially at initial stages when data is lacking), to predict a future trajectory of pandemic growth, and to predict policy impacts due to uncertainty in low-activity populations [15].

Despite the limitations, SEIR-type models can serve as a basis for Bayesian inference. For example, Dehning and colleagues combined Bayesian methodology with an SIR model, which is formulated in terms of coupled ordinary differential equations and enables fast evaluation [16]. Similarly, Ray and colleagues applied a Bayesian extension of the SEIR model, eSIR, to forecast future case counts and study the relative impact of lockdown duration on infections [17]. In the implementation of their model, Markov chain Monte Carlo (MCMC) simulations of the Bayesian model provides a posterior estimation of parameters and prevalence of the compartments in the SIR model. Another approach involves spatial modelling such as in the Global Epidemic and Mobility Model (GLEAM) [1820]. This approach uses a network-based methodology and assumes that the global population can be divided into sub-populations situated around major hubs (e.g., airports and train stations). Within each sub-population, these models apply a framework similar to SEIR models through compartmental representation of the disease.

Medical studies that are relevant to policies

A second stream of related research is comprised of medical studies that explore how the virus is transmitted. In contrast to modeling studies that customarily make use of population-level data, these medical studies pinpoint person-to-person transmission mechanisms through clinical verification within identifiable groups of individuals and/or households. An extensive amount of research has been conducted to describe and measure the airborne transmission of COVID [2124]. In particular, researchers have highlighted the difference between indoor transmission and outdoor transmission, with the former as a predominant driver of disease transmission and the latter as a possible transmission channel [2527]. Research has also highlighted the significant proportion of virus carriers who are asymptomatic [4,28,29]. Additionally, researchers have investigated disease transmission among different population segments. For instance, research shows that the infection rate is generally lower for children than for adults, and that children tend to catch the virus from adults and exhibit a similar level of symptoms as adults rather than disproportionately serving as asymptomatic carriers [3033]. The Bayesian model on which we rely is designed to account for these findings.

Policy studies

A number of studies have investigated the impact of health and non-health policies on COVID spread. This stream of research typically combines models with actual infection and death data to gauge policy effectiveness. For instance, Badr and colleagues find that reduced mobility was associated with reduced COVID growth rates in 20 U.S. counties [34]. Similarly, Bonardi and colleagues study lockdown effects on disease spread and found that reducing movement within countries was an effective control method for developed economies [35]. Meanwhile, other studies point to the lack of effectiveness of certain policies. For instance, Gostic and colleagues find that traveler screening was not particularly effective at containing the pandemic due to a large number of presymptomatic and asymptomatic travelers [36]. Facemask policies are also unlikely to help a country reach required thresholds of disease containment if the policies are loosely enforced and mask wearing remains a voluntary practice [37].

Another topic that has received attention is the impact of school closures on COVID spread. Studies have found mixed results. Some have shown that closing schools in U.S. states led to lower case number and infection rates [38]. Others have shown that, in Canada and Australia, school closures may have limited impact on reducing disease spread [39,40]. A review paper systematically examines research on the effectiveness of school closures at containing coronavirus outbreaks and concludes that school closures do not contribute to the control of the pandemic, especially in comparison to policies that broadly restrict the movement of adults [41].

While most policy studies focus on a specific set of policies or on a particular country, some studies are broader in scope. For example, Hsiang and colleagues investigate policies (non-pharmaceutical interventions) across six countries using econometric methods and conclude that different types of policies, when combined together, led to significantly reduced growth in infections [42]. Flaxman and colleagues study European countries and find that current policy interventions drive the COVID reproduction number below one [43]. Finally, Warne and colleagues have conducted one of the broadest COVID policy studies in existence: they use a stochastic model to examine COVID spread in 158 countries and associate delayed policy responses with rising cases [44].

Methods and data

Prior policy papers generally analyze the effectiveness of COVID control within a geographic area rather than assess how individual policies contribute across jurisdictions to the achievement of COVID control. This study is among the first to study how types of policies–implemented with varying intensity levels–perform across a large number of jurisdictions. In the sections below, we describe the model, data, and distinctive features of the approach.

Epidemiology

The model is based on an exponential growth curve of infection rates by jurisdiction. This curve arises from a parsimonious SIR model in which the susceptible population (S) is assumed to be substantially greater than the infected (I) or recovered (R) population. The model assumes that, in most jurisdictions, just a minority of the population has been infected, and that herd immunity (i.e., more than 50% of the population has been infected) is remote in all jurisdictions. Under these assumptions, the growth in number of infections I is proportional to itself (dI/dt = g I), which gives rise to estimated exponential growth of the number of new infections in each period X(t) = ΔI(t) = egt. In this equation, the variable g represents the growth rate of new infections and is closely related to the reproduction rate R. If R < 1, then g < 0, and the epidemic recedes locally. For R > 1 and g > 0, the number of infections grows exponentially. COVID control requires policies that yield a g below zero.

The reasons we model g rather than R are twofold. First, g is more informative of public-policy choices because it directly relates to key policy concerns, such as how many new infections and deaths must be expected, or how long a certain policy needs to be in place to reduce the number of new infections to a given threshold. Second, g is more readily observable from the data. Estimating R requires making additional assumptions about the time lag between original infection and re-infections by a given individual that are not relevant for policy analysis.

We assume that the growth rate for each jurisdiction i in week t is: (1)

The first term represents the base growth rate by jurisdiction, while the second term represents the growth rate reduction arising from the implemented policies. The base growth rate is represented by the equation git(base) = gi(0/1) + Δgit(id), which consists of two parts. The term gi(0) represents COVID growth prior to the implementation of any policies (e.g., equivalently to R0 in SIR models). After jurisdictions have implemented basic containment policies, gi(0) is replaced by gi(1) (see below for detail on policy coding). The second part, Δgit(id), represents the idiosyncratic growth rate. This term captures any jurisdiction-specific changes over time in COVID growth that cannot be attributed to the policies coded in our database. For this term, we assume a second-order autoregressive process (AR(2)). Mathematical details are provided in the S1 File.

The term ∑pδipt Δgp in Eq (1) represents the effect of further containment policies as coded in the OxCGRT database. The variable δipt = 1 if jurisdiction i had policy p in place in week t, and δipt = 0 otherwise. Δgp ≥ 0 is the estimate for the average effect of each policy.

The growth rate is determined by the weekly change in new infections in logarithms: git = log(Xit)–log(Xi,t-1). Because the number of infections is not directly observable, it is inferred from three different observations by jurisdiction-week: the number of reported COVID cases, the number of reported COVID deaths, and the number of excess deaths. The model considers that reported COVID cases and deaths may be significantly understated, and that the level of underreporting may evolve with changing testing policies. The model also accounts for the lag between infections on the one hand and reported cases and deaths on the other.

Data

We use two sets of input data: policy data to code the variable δipt and epidemiological data to infer the level of new infections Xit. The policy data are based on the OxCGRT database [45], which codes multiple levels of policy responses across many categories for all countries in the world and all U.S. states. The database is publicly available (see links provided at the end of this paper).

Table 1 lists the coding of the levels in the OxCGRT database (the information in this table is copied directly from the Oxford team’s website on GitHub) of policies that related to virus containment. For school closures (category C1), we adapted the coding to include the summer holidays for countries in the northern hemisphere, with level 3 representing closure in all geographies and level 2 in some (because of regional spread of vacations within a jurisdiction). The highest level prevails between the original OxCGRT and the holiday coding (e.g., if OxCGRT codes level 3 for school closures, while holidays are level 2, then we keep the original level 3 coding). Figs 1 and 2 displays the coding of weekly policy by jurisdiction in the model.

thumbnail
Fig 1. COVID policy levels by week based on OxCGRT database.

Policies C1-C6; continued in Fig 2.

https://doi.org/10.1371/journal.pone.0244177.g001

thumbnail
Fig 2. COVID policy levels by week based on OxCGRT database.

Policies C7, C8, H1-H3; continued from Fig 1.

https://doi.org/10.1371/journal.pone.0244177.g002

The model treats the levels cumulatively. For instance, if a jurisdiction is coded as level 2 on policy H2 (virus testing) in a certain week, then we code δipt = 1 for both level 1 and level 2, and δipt = 0 for level 3. In other words, the effect of H2 policies on the overall growth rate git is modelled as Δgp = H2-1 + Δgp = H2-2, with H2-n representing policy level n for category H2.

Because significant policy efforts in essentially all jurisdictions began with C1 (school closing), C2 (Workplace closing), C3 (Cancel public events), and C4 (restrictions on gatherings), we use the implementation of any of these policies at level 1 or above as the criterion for shifting the time-constant part of the base growth rate from gi(0) (free spread of the virus corresponding to R0) to gi(1) (spread of the virus with only basic policies in place). Because C1-C4 are implemented almost simultaneously at an initiating level 1 or greater, they are highly collinear in the estimation of Eq (1)—each has a correlation coefficient of 0.85 or higher with other policies. Therefore, we exclude these policies in the estimation of Δgp. For the same reason we also exclude policy levels that are implemented in fewer than 5% of all policy-weeks. (We also used several different sets of base policies for gi(1) and of excluded variables. The marginal effects for these (unreported) analyses were not significantly different from those reported here.) This results in 23 distinct policy levels that we include in our analysis.

The epidemiological data arise from two sources. COVID case and death data are taken from the Johns Hopkins Coronavirus Resource Center to calculate weekly counts of new cases and deaths by jurisdictions. Excess death data are drawn from The Economist, which reports weekly and monthly total deaths above what was projected based on pre-COVID trends for a limited set of countries and U.S. states. The excess deaths over expected deaths serve as a proxy for the underreporting of COVID-19 deaths.

In our analysis, we include all countries and U.S. states with at least 5 million residents for which The Economist reports excess deaths weekly. Forty jurisdictions meet these criteria. The S1 File contains robustness checks on the criteria for including jurisdictions.

Estimation

Both policy and epidemiological data are used to estimate the effect of various policies on COVID control. The primary variable of interest is Δgp, which represents the marginal effect of each implemented policy, p, on reducing the weekly growth rate of infections. We also report results for git(base), the base growth rate in infections independently of the implemented policies coded in Δgp. Finally, we use the model to make inferences by jurisdiction of weekly new infections Xit and the infection growth rate git.

The Bayesian model generates point estimates (posterior median) and 95% intervals (posterior quantiles) for each of the variables of interest. Eq (1) is linear, and thus would be estimable through ordinary least squares (OLS) regression, but Bayesian estimation confers several advantages. First, the dependent variable (DV) in Eq (1), git = log(Xit)–log(Xi,t-1), is not directly observable because we do not know the actual level of new infections Xit. Therefore, this variable must be estimated probabilistically based on observations of reported COVID cases, reported COVID deaths, and excess deaths. Bayesian modeling is uniquely suited to accomplish this. For the reported case count, Yit(case), we adopt a negative binomial model with rate parameter μ and overdispersion parameter ϕ: (2)

The rate parameter μ represents the expected rate of new cases based on the fraction of total cases αipt(case) that is reported in jurisdiction i with testing policy (H2) p in week t; the fraction of cases ws that is reported with a lag of s weeks; and the actual number of infections Xit. Similar negative binomial models are assumed for reported COVID deaths and excess mortality.

One main advantage of the Bayesian approach is that we can use the observed data to make stochastic inferences (with 95% intervals) about the three parameters α, w, and—most importantly—the number of new infections X, without any further assumptions. The Markov Chain Monte Carlo (MCMC) estimation essentially performs hundreds of simulations of parameter combinations, calculating the likelihood of each combination given the observed data of reported cased, reported deaths, and excess deaths.

A second advantage of the Bayesian model is that it is robust to outliers [46: p.435]. This is important because extreme outliers may influence the inference despite the feature of the negative binomial distribution to provide some protection against outliers by allowing over-dispersion as compared to the Poisson distribution. For instance, in some weeks, a few jurisdictions report very few or no cases due to bureaucratical constraints on disclosure. In the Bayesian model, no distortion arises as a result of this practice through the assumption that an observation is randomly generated with a small probability poutlier, and that the observation is drawn from the distribution represented by Eq (2). The model assumes poutlier = 10−3 for reported COVID cases and deaths (no assumption is necessary for excess deaths because these do not exhibit outliers). In the S1 File we also present the results of robustness checks using outlier probabilities of 10−2 and 10−4.

Third and finally, Bayesian analysis accommodates further constraints and/or stochastic assumptions. For instance, we impose a constraint that policy effects reduce rather than increase infection by assuming Δgp ≥ 0. Other constraints include that the fraction α of cases/deaths that is reported and the lag w take on values between 0 and 1, and that the sum of lag weights w equals 1. Similarly, the estimation accommodates reliance on the AR(2) process for the idiosyncratic term Δgit(id) (see Equation (S3) of the S1 File for further details).

Results, discussion, and implications for policy

For the population of countries and U.S. states included in our analysis, the marginal impact Δgp of each policy p is represented in Fig 3. Each row of the figure represents one of the eleven types of policies reported in the OxCGRT database. The bars represent the median estimate of each policy’s impact on reduction in the weekly average growth rate in infections. Each bar is colored to represent the levels of intensity with which the policies were implemented (see Table 1 for descriptions of the levels). The lines represent the 95% interval around each estimate at the maximum policy level. Overall, Fig 3 shows which of the eleven policies–and the level of intensity of those policies–produces the greatest relative impact on reducing growth in COVID infections across the jurisdictions that we study.

thumbnail
Fig 3. Marginal effect sizes of policy measures on reducing weekly growth rates.

Bars = median estimate by policy level; Lines = 95% intervals for maximum policy level.

https://doi.org/10.1371/journal.pone.0244177.g003

Several important results are evident in Fig 3. First, five of the policies have relatively large impact when fully implemented at their highest levels: Workplace closing, restrictions on internal movement, stay-at-home requirements, public information campaigns, and school closings. Four of the five are associated with lower growth of infections even at relatively low levels, but public information campaigns (H1) are most impactful at the highest recorded level (which in this case is level 2, i.e., coordination across traditional and social media). This policy was implemented early in the pandemic by all jurisdictions in our analysis and is part of the core group (see Figs 1 and 2). Second, constraints on the movements of adults through C2 (workplace closings for all but essential workers), C7 (restrictions on internal movements), and C6 (stay-at-home requirements with exceptions for daily exercise, grocery shopping, and ‘essential’ trips) have the greatest marginal impact at intermediate levels. These results are consistent with prior results from the literature suggesting that control of adult-to-adult transmission is central to COVID remediation. Third, the largest marginal effect of school closing (C1) is at level 2 (closing of some categories, such as universities and/or high schools) while a level 3 policy (closing of all schools) provides and additional but lower marginal benefit. Fourth, testing policy and contact tracing are not strongly associated with COVID controlling for the 40 jurisdictions that we study.

Fig 4 represents the current status at the time of writing (November 22, 2020) of the results of the implemented policies. Several jurisdictions have elevated numbers of new infections Xit. For Italy and Switzerland, new infections are more than 200 per 10,000 people weekly, which is close to the highest numbers of infections in the early stages of the pandemic in Spain, New York, and New Jersey. Growth rates of new infections are highest in Sweden, Colorado, and Pennsylvania. In these jurisdictions, new infections are growing at a weekly rate around or above git = 0.4, which corresponds to an increase of 49% new cases every week (e0.4–1 = 0.49). In other jurisdictions, the growth rate is closer to or below zero (although presently the rate is estimated to be below zero with at least 95% certainty only in Belgium, Chile, Denmark, and France). These differences arise from variation in the policies in place as represented in Figs 1 and 2 as well as compliance, which is driven in part by jurisdiction-specific factors such as population density.

thumbnail
Fig 4. Weekly rate of new infections and their growth by jurisdiction as of November 22, 2020.

Dots = median estimates; Lines = 95% intervals.

https://doi.org/10.1371/journal.pone.0244177.g004

Fig 5 provides insight into compliance. The figure reports on estimates of the base growth rate git(base) by jurisdiction in the hypothetical situation in which only level 1 policies were implemented on school closing, workplace closing, canceling public events, and restrictions on gatherings (see Eq (1)). Especially earlier in the pandemic, Sweden stood out as a country in which these base policies appeared to have had a great impact on lowering growth in infections, resulting in a very low base growth rate gi(1). The model does not, however, offer reasons for this outcome. For example, Sweden’s relatively low population density, weather characteristics, and demographics likely play a role in the result. In recent months, Sweden’s base rate of growth in infection is estimated to have increased, while other jurisdictions such as Denmark and South Africa have estimated git(base) that have declined to levels below that of Sweden. By contrast, Brazil, Chile, Mexico, and Spain have had at least some periods in which implementation of only base policies would likely have not had as strong an impact on COVID control.

thumbnail
Fig 5. Growth rate g(base) after jurisdictions have started to implement one of the base policies (level 1 of C1-C4).

A lower number indicates a higher effectiveness to contain the virus. The total growth rate git of the virus is the number in this chart minus the sum of the effects of implemented policies in Fig 3. Lines = median estimates; Grey bands = 95% intervals

https://doi.org/10.1371/journal.pone.0244177.g005

Fig 6 illustrates the implications of compliance in a jurisdiction for the additional high-impact policies that must be implemented above the core group to reduce COVID growth below zero. To read the figure, please begin by examining the g(base) for each category of jurisdiction: the most compliant 10%; the median by compliance; and the least compliant 10%. For jurisdictions in each of these categories, the figure shows how much reduction in the growth rate g has been accomplished on average through implementation of the core policies, which have been widely adopted across jurisdictions because of their social tolerability. These core policies include cancellation of public events (C3–2), restricting gatherings to fewer than 100 people (C4–3), recommending stay-at-home behavior (C6–1), recommending no travel between regions/cities (C7–1), implementing a partial international travel ban (C8–3), instituting coordinated information campaigns (H1–2), and COVID testing and tracing (H2–2 & H3–2), in addition to the base policies (C1-C4 level 1).

thumbnail
Fig 6. Effects of a set of core policies on weekly growth rate g by quantile of g(base) among all jurisdictions.

Core policies include the cumulative effects of C3–2, C4–3, C6–1, C7–1, C8–3, H1–2, H2–2, and H3–2. Dots = median estimates; Lines = 95% intervals.

https://doi.org/10.1371/journal.pone.0244177.g006

The figure shows that the set of core policies lead to COVID control only for jurisdictions with strong policy compliance (i.e., the lowest g(base)). In contrast, for the median jurisdiction, the core policies are insufficient for driving COVID growth below zero despite having a strong effect, i.e., they reduce growth from an estimated 1.3 cases per week corresponding to an increase in virus spread by e1.3–1 = 270% to an estimated 0.4 cases per week corresponding to a growth of e0.4–1 = 49%. Hence, for 90% of jurisdictions, the core policies are insufficient to achieve COVID control. Almost all jurisdictions must implement more stringent policies to achieve a further reduction of the weekly growth rate g to below zero. Based on the marginal effects in Fig 3, the additional high-impact policies (with median estimated effectiveness Δgp above 0.1) are:

  • Targeted workplace closings (C2–2; Δgp = 0.20)
  • Full workplace closings (C2–3; Δgp = 0.14, on top of C2–2)
  • Stay-at-home requirement with exceptions (C6–2; Δgp = 0.14)
  • Targeted school closings (C1–2; Δgp = 0.14)

The next two most effective policies that must be implemented in addition to the core policies are (with median estimated effectiveness Δgp between 0.05 and 0.1):

  • Full school closings (C1–3; Δgp = 0.07, on top of C1–2)
  • Restrictions on gatherings below 10 (C4–4; Δgp = 0.07)

Each of these policies has significant societal costs. Unfortunately, however, unless governments can compel greater compliance with core policies, jurisdictions must implement several of these additional policies to ensure that virus spread does not grow exponentially.

As with any study, these results must be interpreted within the limitations of the methods used. One limitation is that we need to make several assumptions as described in the Methods section about disease epidemiology and the impact of government policy. In the S1 File, we have included several robustness checks that demonstrate that our results are relatively insensitive within reasonable parameters to specific assumptions. Second, because this is an observational study, it is inherently limited because the findings may be biased by variables that are not included in the study. For instance, the relatively limited impact of testing (H2) and contact tracing (H3) in Fig 3 could arise from the lifting of policies others than those reported in the Oxford database around the same time that H2 and H3 policies were implemented. While we strive to limit the impact of omitted-variable bias through reliance on 23 different policy levels, the findings must be interpreted with caution. Overall, the results should be interpreted with these limitations in mind and should be relied on only in conjunction with results from other studies, including computational models and controlled experiments such as those described in our review of the literature.

Conclusion

The analysis generates several important insights. First, important differences arise across jurisdictions in infection levels, death rates, policy implementation, and compliance with policies. These differences have major implications for COVID control. The portfolio of the eleven types of policies–implemented with different levels of intensity–necessary to drive COVID growth rate below zero depends on the jurisdiction’s COVID burden and compliance, which reflects behavioral and demographic characteristics of the jurisdictions. Second, we estimate that a core set of socially tolerable policies lead to COVID control only in those jurisdictions that have unusually high levels of compliance. The socially tolerable core policies alone are meaningful and significant, but insufficient by themselves for preventing escalating growth in infections in 90% of the jurisdictions analyzed. For these jurisdictions, one or more from a set of additional high-impact but difficult-to-tolerate policies must be implemented to achieve COVID control. Third, the impact of testing and contact tracing has been lower than the impact of other policies. Fourth, for the jurisdictions covered in this analysis, the policies with the greatest marginal impact for achieving COVID control mainly involve restrictions on adults through workplace closings and stay-at-home requirements, although targeted school closings are also in the group of additional high-impact policies. Altogether, the analysis indicates that, in all but a few highly compliant jurisdictions, relatively significant social costs must be incurred to reduce COVID growth below zero. The model points to significant opportunities for cultivating deeper understanding of the drivers of compliance and of variation in the impact of policies, which are potentially rewarding topics for future research.

Supporting information

S1 Fig. Newly identified COVID cases, COVID deaths, total deaths, and infections per week (log scale).

Dots = reported; X = outlier; Solid lines = model fit; Dashed lines = 95% intervals.

https://doi.org/10.1371/journal.pone.0244177.s001

(DOCX)

S2 Fig. Sensitivity of policy effect estimates (as reported in Fig 2 of the main text) to different model specifications.

Dots = median estimates; Lines = 95% intervals; Red = Base estimate outside of 95% posterior interval of changed specification.

https://doi.org/10.1371/journal.pone.0244177.s002

(DOCX)

S3 Fig. Percentage of weekly reported COVID cases and deaths within 95% interval of out-of-sample predictions from a model using input data up to 6 weeks prior to present.

https://doi.org/10.1371/journal.pone.0244177.s003

(DOCX)

S4 Fig. Out of sample estimates of newly identified COVID cases and deaths (log scale), with reported numbers.

Dots = reported; Solid lines = model prediction; Dashed lines = 95% intervals.

https://doi.org/10.1371/journal.pone.0244177.s004

(DOCX)

S1 Table. Posterior median and 95%-interval estimates for key variables of interest, as well as Bayesian estimate statistics neff and Rhat.

https://doi.org/10.1371/journal.pone.0244177.s005

(DOCX)

S2 Table. Variance inflation factors for the policy fixed effects δipt by policy p.

https://doi.org/10.1371/journal.pone.0244177.s006

(DOCX)

S1 File. Appendix with methodological details and robustness checks.

https://doi.org/10.1371/journal.pone.0244177.s007

(DOCX)

References

  1. 1. Wu JT, Leung K, Bushman M, Kishore N, Niehus R, de Salazar PM, et al. Estimating clinical severity of COVID-19 from the transmission dynamics in Wuhan, China. Nat Med. 2020;26: 506–510. pmid:32284616
  2. 2. Rossman H, Keshet A, Shilo S, Gavrieli A, Bauman T, Cohen O, et al. A framework for identifying regional outbreak and spread of COVID-19 from one-minute population-wide surveys. Nat Med. 2020. pmid:32273611
  3. 3. Cohen J, Kupferschmidt K. Labs scramble to produce new coronavirus diagnostics. Science (80-). 2020;367: 727. pmid:32054740
  4. 4. Gandhi M, Yokoe DS, Havlir DV. Asymptomatic transmission, the achilles’ heel of current strategies to control Covid-19. New England Journal of Medicine. Massachussetts Medical Society; 2020. pp. 2158–2160. https://doi.org/10.1056/NEJMe2009758 pmid:32329972
  5. 5. Wilson N, Kvalsvig A, Barnard L, Baker M. Case-Fatality Risk Estimates for COVID-19 Calculated by Using a Lag Time for Fatality. Emerg Infect Dis. 2020;26. pmid:32168463
  6. 6. Anastassopoulou C, Russo L, Tsakris A, Siettos C. Data-based analysis, modelling and forecasting of the COVID-19 outbreak. PLoS One. 2020;15: e0230405. pmid:32231374
  7. 7. Bastos SB, Cajueiro DO. Modeling and forecasting the early evolution of the Covid-19 pandemic in Brazil. arXiv Prepr. 2020;2003. pmid:33173127
  8. 8. Calafiore GC, Novara C, Possieri C. A Modified SIR Model for the COVID-19 Contagion in Italy. arXiv Prepr. 2020. pmid:33132739
  9. 9. Dukic V, Lopes HF, Polson NG. Tracking Epidemics With Google Flu Trends Data and a State-Space SEIR Model. J Am Stat Assoc. 2012;107: 1410–1426.
  10. 10. Lekone PE, Finkenstädt BF. Statistical Inference in a Stochastic Epidemic SEIR Model with Control Intervention: Ebola as a Case Study. Biometrics. 2006;62: 1170–1177. pmid:17156292
  11. 11. Prem K, Liu Y, Russell TW, Kucharski AJ, Eggo RM, Davies N, et al. The effect of control strategies to reduce social mixing on outcomes of the COVID-19 epidemic in Wuhan, China: a modelling study. Lancet Public Heal. 2020. (20)30073–6. pmid:32220655
  12. 12. Peng L, Yang W, Zhang D, Zhuge C, Hong L. Epidemic analysis of COVID-19 in China by dynamical modeling. arXiv Prepr. 2020;2002. pmid:32987516
  13. 13. Gatto M, Bertuzzo E, Mari L, Miccoli S, Carraro L, Casagrandi R, et al. Spread and dynamics of the COVID-19 epidemic in Italy: Effects of emergency containment measures. Proc Natl Acad Sci U S A. 2020;117. pmid:32327608
  14. 14. Moghadas S, Fitzpatrick M, … PS. The implications of silent transmission for the control of COVID-19 outbreaks. Proc Natl Acad Sci U S A. 2020;117: 17513–17515. pmid:32632012
  15. 15. Ellison G. Implications of Heterogeneous SIR Models for Analyses of COVID-19. NBER Work Pap Ser. Cambridge, MA; 2020 Jun. Report No.: w27373. https://doi.org/10.3386/w27373
  16. 16. Dehning J, Zierenberg J, Spitzner F, … MW-, 2020 U. Inferring change points in the spread of COVID-19 reveals the effectiveness of interventions. Science (80-). 2020;369.
  17. 17. Ray D, Salvatore M, …, Mukherjee B.Predictions, role of interventions and effects of a historic national lockdown in India’s response to the COVID-19 pandemic: data science call to arms. Harvard Data Sci Rev. 2020. pmid:32607504
  18. 18. Balcan D, Colizza V, Gonçalves B, Hu H, Ramasco JJ, Vespignani A. Multiscale mobility networks and the spatial spreading of infectious diseases. Proc Natl Acad Sci U S A. 2009. pmid:20018697
  19. 19. Chinazzi M, Davis JT, Ajelli M, Gioannini C, Litvinova M, Merler S, et al. The effect of travel restrictions on the spread of the 2019 novel coronavirus (COVID-19) outbreak. Science (80-). 2020; eaba9757. pmid:32144116
  20. 20. Zhang Q, Sun K, Chinazzi M, Piontti APY, Dean NE, Rojas DiP, et al. Spread of Zika virus in the Americas. Proc Natl Acad Sci U S A. 2017;114: E4334–E4343. pmid:28442561
  21. 21. Asadi S, Bouvier N, Wexler AS, Ristenpart WD. The coronavirus pandemic and aerosols: Does COVID-19 transmit via expiratory particles? Aerosol Sci Technol. 2020;54: 635–638. pmid:32308568
  22. 22. Cascella M, Rajnik M, Cuomo A, Dulebohn SC, Di Napoli R. Features, Evaluation and Treatment Coronavirus (COVID-19). StatPearls. StatPearls Publishing; 2020.
  23. 23. Setti L, Passarini F, De Gennaro G, Barbieri P, Perrone MG, Borelli M, et al. Airborne Transmission Route of COVID-19: Why 2 Meters/6 Feet of Inter-Personal Distance Could Not Be Enough. Int J Environ Res Public Health. 2020;17: 2932. pmid:32340347
  24. 24. Wilson N, Corbett S, Tovey E. Airborne transmission of covid-19. BMJ. 2020;370: m3206. pmid:32819961
  25. 25. Coleman K, Nguyen T, Reports SY-S, 2018 U. Bioaerosol sampling for respiratory viruses in Singapore’s mass rapid transit network. Nat Sci Reports. 2018;8. pmid:30504827
  26. 26. Morawska L, Tang JW, Bahnfleth W, Bluyssen PM, Boerstra A, Buonanno G, et al. How can airborne transmission of COVID-19 indoors be minimised? Environment International. Elsevier Ltd; 2020. p. 105832. https://doi.org/10.1016/j.envint.2020.105832 pmid:32521345
  27. 27. Nishiura H, Kobayashi T, … TM-… journal of infectious, 2020 U. Estimation of the asymptomatic ratio of novel coronavirus infections (COVID-19). Int J Infect Dis. 2020;94.
  28. 28. Hu Z, Song C, Xu C, Jin G, Chen Y, Xu X, et al. Clinical characteristics of 24 asymptomatic infections with COVID-19 screened among close contacts in Nanjing. Sci China Life Sci. 2020;63: 706–711. pmid:32146694
  29. 29. Mizumoto K, Kagaya K, Zarebski A, Chowell G, Kenji M, Katsushi K, et al. Estimating the asymptomatic proportion of coronavirus disease 2019 (COVID-19) cases on board the Diamond Princess cruise ship, Yokohama, Japan, 2020. Eurosurveillance. 2020;25.
  30. 30. Cai J, Xu J, Lin D, Yang Z, Xu L, Qu Z, et al. A Case Series of children with 2019 novel coronavirus infection: clinical and epidemiological features. Clin Infect Dis. 2020. pmid:32112072
  31. 31. Li W, Zhang B, Lu J, Liu S, Chang Z, Cao P, et al. The characteristics of household transmission of COVID-19. Clin Infect Dis. 2020. pmid:32301964
  32. 32. Posfay-Barbe KM, Wagner N, Gauthey M, Moussaoui D, Loevy N, Diana A, et al. COVID-19 in children and the dynamics of infection in families. Pediatrics. 2020;146. pmid:32457213
  33. 33. Qiu H, Wu J, Hong L, Luo Y, Song Q, Chen D. Clinical and epidemiological features of 36 children with coronavirus disease 2019 (COVID-19) in Zhejiang, China: an observational cohort study. Lancet Infect Dis. 2020;20: 689–696. pmid:32220650
  34. 34. Badr H, Du H, Marshall M, Dong E, Squire M, Gardner L. Association between mobility patterns and COVID-19 transmission in the USA: a mathematical modelling study. Lancet Infect Dis. 2020. pmid:32621869
  35. 35. Bonardi J-P, Gallea Q, Kalanoski D, Lalive R. COVID ECONOMICS VETTED AND REAL-TIME PAPERS Fast and local: How lockdown policies affect the spread and severity of covid-19 COVID ECONOMICS. Covid Econ. 2020;23: 325–351.
  36. 36. Gostic K, Gomez A, Mummah R, Kucharski A, Lloyd-Smith J. Estimated effectiveness of symptom and risk screening to prevent the spread of COVID-19. Elife. 2020;9. pmid:32091395
  37. 37. Betsch C, Korn L, Sprengholz P, Felgendreff L, Eitze S, Schmid P, et al. Social and behavioral consequences of mask policies during the COVID-19 pandemic. Proc Natl Acad Sci U S A. 2020;117: 21851–21853. pmid:32820078
  38. 38. Auger KA, Shah SS, Richardson T, Hartley D, Hall M, Warniment A, et al. Association Between Statewide School Closure and COVID-19 Incidence and Mortality in the US. JAMA. 2020;324: 859. pmid:32745200
  39. 39. Abdollahi E, Haworth-Brockman M, Keynan Y, Langley JM, Moghadas SM. Simulating the effect of school closure during COVID-19 outbreaks in Ontario, Canada. BMC Med. 2020;18. pmid:32709232
  40. 40. Chang SL, Harding N, Zachreson C, Cliff OM, Prokopenko M. Modelling transmission and control of the COVID-19 pandemic in Australia. arXiv Prepr. 2020. pmid:33177507
  41. 41. Viner RM, Russell SJ, Croker H, Packer J, Ward J, Stansfield C, et al. School closure and management practices during coronavirus outbreaks including COVID-19: a rapid systematic review. The Lancet Child and Adolescent Health. Elsevier B.V.; 2020. pp. 397–404. https://doi.org/10.1016/S2352-4642(20)30095-X pmid:32272089
  42. 42. Hsiang S, Allen D, Annan-Phan S, Bell K, Bolliger I, Chong T, et al. The effect of large-scale anti-contagion policies on the COVID-19 pandemic. Nature. 2020;584: 262–267. pmid:32512578
  43. 43. Flaxman S, Mishra S, Gandy A, Unwin HJT, Mellan TA, Coupland H, et al. Estimating the effects of non-pharmaceutical interventions on COVID-19 in Europe. Nature. 2020;584: 257–261. pmid:32512579
  44. 44. Warne DJ, Ebert A, Drovandi C, Mira A, Mengersen K. Hindsight is 2020 vision: Characterisation of the global response to the COVID-19 pandemic. medRxiv. 2020. pmid:33287789
  45. 45. Hale T, Angrist N, Kira B, Petherick A, Phillips T, Webster S. Variation in government responses to COVID-19. Blavatnik Sch Gov Work Pap. 2020. Report No.: 31.
  46. 46. Gelman A, Carlin J, Stern H, Dunson D, Vehtari A, Rubin D. Bayesian data analysis. 3rd ed. Boca Raton, FL, USA: CRC Press; 2013.