Measuring the efficacy of a vaccine during an epidemic

Antonio Scala; Pierpaolo Cavallo

doi:10.1371/journal.pone.0290652

Abstract

The urgency to develop vaccines during the COVID-19 pandemic has resulted in the acceleration of clinical trials. Specifically, a broad spectrum of efficacy levels has been reported for various vaccines based on phase III cohort studies. Our study demonstrates that conducting large cohort phase III clinical trials during the peak of an epidemic leads to a significant underestimation of vaccine efficacy, even in the absence of confounding factors. Furthermore, we find that this underestimation increases with the proportion of infectious individuals in the population during the experiment and the severity of the epidemic, as measured by its basic reproduction number.

Citation: Scala A, Cavallo P (2023) Measuring the efficacy of a vaccine during an epidemic. PLoS ONE 18(9): e0290652. https://doi.org/10.1371/journal.pone.0290652

Editor: Tiago Pereira, Universidade de Sao Paulo, BRAZIL

Received: December 31, 2022; Accepted: August 11, 2023; Published: September 14, 2023

Copyright: © 2023 Scala, Cavallo. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within the manuscript. However, the matlab file used to make calculations and generate the images of the paper are already available on zenodo, with doi 10.5281/zenodo.7881717 and link https://doi.org/10.5281/zenodo.7881717.

Funding: The author(s) received no specific funding for this work.

Competing interests: The authors have declared that no competing interests exist.

Introduction

The urgency driven by the COVID-19 pandemic has resulted in the implementation of clinical trials with procedures that adapt to the extraordinary circumstances [1] and the establishment of unprecedented public-private partnerships [2]. In particular, we have observed the case of vaccines that have reported widely different efficacies [3], varying from the ∼95% of Pfizer and Moderna (mRNA based), to the ∼70% of Astra-Zeneca or the ∼66% of Johnson & Johnson (viral vector-based). While mRNA-based vaccines, such as Pfizer-BioNTech and Moderna vaccines, instruct our cells directly to produce spike proteins, viral vector vaccines like AstraZeneca and Johnson & Johnson employ a harmless viral vector to deliver the genetic instructions for spike protein production to our cells. Once spike proteins are produced, our immune system recognizes them as foreign and mounts a response. This includes the production of antibodies that can bind to and neutralize the spike proteins, as well as the activation of T cells. These immune responses provide immunity against the virus. Given that all these vaccines induce an immune response by stimulating spike protein production, it is worth considering whether the variability in results could be influenced by differences in experimental conditions, such as the fraction of infectious individuals and the presence of variants. In this manuscript, we present findings that demonstrate how measuring vaccine efficacy at different stages of an epidemic’s progression can lead to a significant underestimation of the efficacy.

Vaccine efficacy is defined as one minus some measure of relative risk; according to the risk considered, several measures can be defined: efficacy for susceptibility to disease, for colonization, for progression, pathogenicity, infectiousness, indirect effects, population-level effects etc [4]. These measures require specialised and accurate datasets, sometimes with detailed information on the single contact experienced by the experimental cohorts. A vaccine of efficacy ϵ decreases the transmission rate by a factor δ = 1 − ϵ, i.e. a vaccinated person has a probability δ times lower of getting infected when coming into contact with an infectious individual; δ can be also be called the relative risk of vaccinated individuals [4]. We will employ a simpler characterization of the vaccine efficacy ϵ defined in terms of the transmission rate β of the epidemic. Thus, the transmission rate for vaccinated persons lowers from β to β^V = δ ⋅ β [5].

On the other hand, vaccine effectiveness η measures the real-world performance of a vaccine [6, 7], in contrast with efficacy that can be defined as the performance of an intervention under ideal and controlled circumstances. Factors concurring in a deviation of effectiveness from efficacy are multifaceted, and the implementation of effectiveness studies (especially challenging low- and middle-income countries) is affected by several confounding factors like age, socio-demographic factors (ethnicity/religion), geographical location, chronic disease and/or comorbidities and socio-economic status [8]. Thus, the same vaccine can have two different efficacies if tested or administered in two regions with different social contact patterns. In particular, a factor that has been mostly disregarded in large cohort (i.e. phase III) studies is the impact of the fraction of infectious individuals during the trial since in the past it had never occurred to develop and test a vaccine during an ongoing epidemic.

A key metric for the impact of a pandemic is the basic reproduction number , measuring the expected number of people an individual can infect; can be calculated in terms of the transmission probability β and of the average lifetime τ of the infectious state as [9]. The basic reproduction number allows us to estimate the herd immunity threshold (HIT), i.e. the fraction of immune individuals beyond which no epidemic overburst can happen [9]. The efficacy ϵ is paramount for estimating the effective fraction of people ρ^V = ρ^*/ϵ to reach the HIT: the lower the efficacy, the higher the fraction of individuals to vaccinate.

The efficacy is not known a priori, but must be estimated through an experimental procedure. Overestimating ϵ would underestimate ρ^V, with the danger of not reaching the HIT at the end of the vaccination campaign. Underestimating ϵ ensures that the fraction of vaccinated people is beyond the HIT; however, it increases both the costs and the duration of a vaccination campaign and—in extreme cases—it can lead to an estimate of the number of individuals to vaccinate beyond any practical possibility. As an example, if the fraction of kids in a population is ρ^kids but the vaccine cannot be administered to kids, ρ^V cannot be achieved if it is higher than 1 − ρ^kids.

Reported efficacies are a measure of the reduction of the incidence of and outcome in a vaccinated group compared to an unvaccinated group under optimal conditions in a clinical trial. However, what happens if clinical trials are performed on large cohorts and during an epidemic so that it is possible that optimal conditions cannot be strictly enforced? As noted by Hallorane et al [10], to avoid that equivalent populations with the same transmission conditions could yield different efficacy estimates, the amount of exposure to infection should be taken into account either by study design or by mathematical modelling.

To isolate the effect of pursuing clinical trials during an ongoing epidemic, we will consider the theoretical case where no confounding factors [6, 7] intervene in the effectiveness—measured as the experimental ratio of infected individuals in a vaccinated and a placebo cohort—showing in long trials performed during an ongoing epidemic the vaccine effectiveness η underestimates the vaccine efficacy ϵ; such underestimation grows both with the fraction of infectious individuals i present in the population during the experiment and with the severity of the infection.

Methods

Vaccine efficacy

When the frequency of infective events in the susceptible individuals depends on the number of already affected individuals [11], the interpretation of the estimates of a vaccine efficacy can vary depending on the assumptions about the underlying dynamics [4]. Let’s assume that, to perform a double-blind evaluation of a vaccine’s efficacy, individuals have been divided into two cohorts V (the ones that have received the vaccines) and P (the ones that have received the placebo). Let’s also assume that the experimental protocol ensures that: (i) the individual in the cohorts are not in reciprocal contact (the ideal case is when infectious individuals in the cohorts remain reciprocally uncorrelated during the experiment); (ii) the infectious dynamics of the cohorts does not influence significantly the ongoing epidemics, i.e the size of the group is much smaller than the population and the observation time is much lower than the total time for the epidemic to evolve. Under these assumptions, infections come only from contacts with infectious individuals outside the cohorts. Thus, assuming full mixing, the probability of meeting an infectious individual is proportional to the fraction i of infectious individuals in the whole population, and the evolution of the fraction of susceptible individuals (i.e. not yet infected) in the P, V cohorts can be written as: (1) where the transmission probabilities are β^P = β for the placebo cohort and β^V = δ ⋅ β for the vaccine cohort, where δ = 1 − ϵ. Both equations can be solved yielding the solutions (2) where we indicate with the attack rate of the infection for the period [t, t + T], i.e the cumulative fraction of infectious individuals [9] during the trial. The corresponding attack rates for the cohorts will be c_P = 1 − s_P and c_V = 1 − s_V; thus, we can rewrite Eq 9 as (3) that tells us that the observed effectiveness η will depend on the attack rate relative to the observation period; notice that such expression is in accordance with the results of [12]. For small values of βc, it is possible to expand Eq 3: (4) i.e. there is already a negative correction to the estimate of ϵ by η that is proportional to δ even for small values of c (i.e. when the average number of infectious is small). Also the expansion to the second order (5) retains the same behaviour, since the quadratic term is still negative up to very low efficacy ϵ = 0.5 and the corrections decrease proportionally to δ: the higher the efficacy, the better the effectiveness η estimates the efficacy ϵ.

SIR model

To estimate c, it would be necessary to have accurate data on the fraction of infectious individuals during an epidemic, like the one obtained by testing campaigns. In cases like COVID19, where data are scarce and the understanding of the epidemic is still an ongoing process, it is useful to rely on mathematical models whose parameters are tuned to the epidemic’s dynamics. For its simplicity and for the few parameters needed, we will use the basic SIR model. While the SIR model is not the best model for accurate scenario forecasting, it captures the overall behaviour of an epidemic with a minimal amount of parameters; thus, it is often used in the first stages of an epidemic when data are still scarce. On the same footing, we will use the SIR model to have an order-of-magnitude estimate of the effects on the measurement of the effectiveness during an ongoing epidemic.

In the SIR model, the population is divided into three groups S,I,R corresponding to different stages of an infection: S corresponds to susceptible individuals, I to infectious and R to recovered individuals. Indicating with lowercase letters (i.e. s, i, r) the fractions of individuals in a given class, the epidemic is described by the equations (6) where β is the infection rate and τ is the average duration of the infectious period.

For the SIR model, since [13], it is possible to derive the closed solution for a free epidemic starting from s(t = 0) = 1, r(t = 0) = 0; thus, since i is maximum when and at this value , we can explicitly calculate the value of i^max from i + s + r = 1 (7) showing that in SIR models the maximum fraction of infectious grows as expected with the basic reproduction number following a simple relation with .

Stochastic estimates of the efficiency

While deterministic equations for epidemic dynamics can be a good approximation when the population is large and as soon as there is an extensive number (even if the fraction is small) of infectious [9], in the case of medical experiments cohorts are seldom large enough to disregard statistical fluctuations in the observations. Apart from particular cases like systems with critical points [14], relative fluctuations for a system of n individuals are of the order . Thus, while for equations like SIR’s—describing populations of a size of the order of the inhabitants of a nation—we can disregard fluctuations and we can thus consider i as a good proxy for the evolution of the fraction of infectious, Eq 1 does not allow to check for the importance of fluctuations in the experimental setting when the number of cohorts’ patients n is not so large. As an example, cohorts of size n ∼ 10000 are expected to yield relative errors of order ∼1%.

To estimate such statistical fluctuations, we employ a simplified stochastic approach. Since the V, P experimental cohorts consist of independent and uncorrelated individuals, the infection rates β^Xi, X ∈ {V, P} of Eq 1 can be interpreted as independent Poisson rates where each individual in the cohorts has a probability −β^Xi per unit time to become infected. In a time interval Δt small enough that i can be considered constant, the number of infections suffered by a population of S^X individuals will thus follow a Poisson distribution of mean β^XiS^XΔt; thus, the infectious dynamics for the experimental cohorts can be simulated as (8) where X ∈ {V, P} and PoissRand(x) generates a random integer number Poisson distributed with rate parameter x. Such an approach has been applied to estimate the fluctuations reported in Fig 1.

Download:

Fig 1. Measured effectiveness η (Eq 9) versus the attack rate c.

The epidemic is modelled with a SIR model with basic reproduction number and mean infectious period τ = 15 days corresponding to a transmission rate β = 0.2 days⁻¹. The continuous black line corresponds to the expected values of η (Eq 3) for trials of a duration T = 2 months and real efficacy ϵ = 0.90. Curves are obtained by varying the initial time t of the trial; thus, each c corresponds to a period [t, t + T]. Lower values of c correspond to the initial and final phases of the epidemics where the fraction of infectious individuals is low, while high values of c correspond to experiments performed near the peak of the epidemic. We observe that η is affected by a systematic error (i.e. η < ϵ) that makes it underestimate the real efficacy ϵ; when the fraction of infectious individuals is high, the error is larger, while when it is low, η ≈ ϵ and the error is proportional to βc (see Eq 4). To evaluate the statistical errors, we model the process of getting infected by a stochastic process (Eq 8) and simulate possible values of η for cohorts of n = 4 × 10⁴ individuals, i.e. of a size of the same order of the Pfizer trial [16]). As expected, the results of the stochastic simulations (red dots in the figure) fall in a region with a distance of order 10⁻² around the theoretical curve of Eq 3, i.e. a region of order as expected for a trial with cohorts of independent, non-interacting individuals.

https://doi.org/10.1371/journal.pone.0290652.g001

Results

The effectiveness η of the vaccine is measured by confronting the infections occurring into two observed groups (also called cohorts in the medical language), one that has been vaccinated (cohort V of size N_V) and one that got a placebo (cohort P of size N_P) [5]. The distribution of the traits (age, census, medical history, etc) of both cohorts must be representative of the whole population; moreover, (i) the individuals of both cohorts should be distributed in the population so as not to have contacts among themselves (to avoid spurious correlations), (ii) the observation period T should be long enough to have a statistically significant number of observed cases of infections. Thus, the effectiveness η is estimated as the proportion of persons in the placebo group of a vaccine trial who would not have become ill if they had received the vaccine [5]. Let us indicate with A^V and A^P the number of cases in the vaccinated and placebo cohorts, respectively, at the end of the study. Suppose that we are analysing a large trial (like a phase III study) where, given the number of participants, it is not possible to have detailed information about their contacts. If we indicate with c_P = A_P/N_P and c_V = A_V/N_V the attack rates (or cumulative incidence), i.e. the fraction of individuals that get infectious during the trial, the vaccine efficacy can be expressed as [4] (9) however, we must bear in mind that interpreting efficacy estimates is a multifaceted issue [4].

Notice that Eq 9 could be a good estimate of the vaccine efficacy ϵ if the observed cohorts are under controlled clinical trial conditions [4]; in the case of large phase III studies, this is not the case: in particular, if phase III studies are performed during an epidemic, the measure of η could depend on the fraction of infectious individuals in the population.

The effectiveness η aims to be a proxy (in statistical language—an estimator) of the real efficacy ϵ of the vaccine. In the following, due to the nature of a phase III experiment (large numbers of individuals not subject to clinical trial conditions), we will assume that cohort individuals are in contact with the infected population. Since the number of individuals in the cohorts is much smaller than the population, we will also assume that the individuals in the V, P cohorts are uncorrelated as required from the experimental protocol. Finally, we will assume that the dynamics of the cohorts do not influence significantly the ongoing epidemics: this is true if the size of the group is much smaller than the population and if the observation time is much lower than the total duration of the epidemic. Under such assumption, it is possible to derive an explicit formula (Eq 3) for the final values of the effectiveness η in terms of the relative attack rate c, of the transmission rate β and of the relative risk δ. Since the attack rate can be expressed as , we can see that the key drivers are the length T of the experiment and the average fraction of infectious during the period T.

In the following, we will show the results for SIR models with parameters in the range of COVID19 estimates; in particular, we will assume that the infectious period is τ = 15 days and is in the range [2.5, 6.0] [15]; however, since Eq 3 does not depend on the details of the dynamics, the results are expected to be robust in respect of the epidemic model employed.

In Fig 1, we show how the expected estimates of η (Eq 3) for a real efficacy ϵ = 0.90 decrease as a function of the attack rate c. The basic reproduction number is fixed to be , while the duration of the trial is fixed to be T = 4τ, i.e. a period of ≈2 months. Lower values of c correspond to the initial and final phases of the epidemics where , while high values of c correspond to experiments performed near the peak of the epidemic. We observe that η tends to underestimate ϵ more when the fraction of infectious individuals is high, while η ≈ ϵ in the initial phases where ; in particular, in this regimes, the corrections to η (i.e. the systematic errors introduced by using Eq 9) are small and proportional to the attack rate during the trial (see Eq 4).

When the number of individuals is small, the process of getting infected is better described by a stochastic process. We thus perform stochastic simulations of the experiments (see Methods) for cohorts of n = 4 × 10⁴ individuals, i.e. of a size of the same order of the Pfizer trial [16]). In Fig 1 we show as red dots the results of stochastic simulations of the trial; the results of the stochastic simulation fall in a region with a relative distance of order 10⁻² around the theoretical curve of Eq 3, i.e in a region of order as expected for a trial with cohorts of independent, non-interacting individuals.

Since the maximum number of infectious individuals is an increasing function of (see Eq 7), the maximum attainable value of c also increases with ; thus, the worst estimated effectiveness η^min (i.e. at the infectious peak) must also be a decreasing function of . In Fig 2, we show that this is the case by plotting η^min as a function of for ϵ = 0.90, 0.93, 0.96 and for a duration of the experiment of 4τ, i.e. T ≈ 2 months. Notice that, if the vaccine has a higher efficacy ϵ, then η better estimates it: as an example, for (a value that has been estimated for COVID19 in France [17]), to an efficacy ϵ = 0.96 corresponds an effectiveness as low as η^min ≈ 0.90 (i.e. Eq 9 introduces a systematic error up to ∼6%), while to an efficacy ϵ = 0.90 corresponds an effectiveness as low as η^min ≈ 0.77 (i.e. Eq 9 introduces a systematic error up to ∼15%).

Download:

Fig 2. Minimum efficacy vs basic reproduction number.

According to Eq 3, the measured effectiveness η (Eq 9) reaches a minimum η^min for clinical trials near the epidemic peak. The figure reports the theoretical values of the worst effectiveness’ estimate η^min versus the basic reproduction number when modelling the epidemic with a SIR of mean infectious period τ = 15 days and considering clinical trials of length T = 2 months. The three curves correspond to a true efficacy of ϵ = 0.90 (continuous line), ϵ = 0.93 (dashed line) and ϵ = 0.96 (dotted line). The curves show that the lower a vaccine’s efficacy ϵ, the worse it is underestimated by the effectiveness η (Eq 9).

https://doi.org/10.1371/journal.pone.0290652.g002

Finally, in Fig 3, we show an example of the time-dependence of the measured effectiveness η in the case of a SIR model with and τ = 15 days. We then consider η(t) as the effectiveness measured as Eq 9 on a series of T = 2 months trials starting at different times t for a vaccine with true efficacy ϵ = 90%. As expected, η(t)∼ϵ when the initial fraction i of infected is very low, η(t) decreases when i grows and reaches before during the infection peak. In fact, for the same number of infected at the beginning of a trial, the measured effectiveness will be lower if the epidemic is growing since the attack rate in Eq 9 will be higher.

Download:

Fig 3. Measured effectiveness and fraction of infected individuals versus time.

We consider an SIR model with and τ = 15 days. Upper panel: the effectiveness η(t) measured as Eq 9 on a series of T = 2 months trials starting at different times t for a vaccine with true efficacy ϵ = 90%. As expected, η(t)∼ϵ when the initial fraction i of infected is very low. η(t) decreases as i grows and reaches a minimum during the infection peak. In fact, for the same number of infected at the beginning of a trial, the measured effectiveness will be lower if the epidemic is growing since the attack rate in Eq 9 will be higher.

https://doi.org/10.1371/journal.pone.0290652.g003

Discussion

The rush to develop COVID-19 vaccines has compelled governments and developers to establish new standards for robust human clinical trials. It has become evident that methodological challenges in these trials can result in unrepresentative data, and communication errors can further contribute to vaccine hesitancy [18].

Assessing vaccine efficacy necessitates a comprehensive understanding of the trial participants’ health status and their contacts. However, this level of information is rarely available, except in early-stage phase I and II studies, where smaller groups in controlled settings allow for more in-depth insights into the vaccine’s effects. These studies can help separate the indirect effects caused by a growing number of infections in the overall population [19] and distinguish the vaccine’s efficacy in protecting individuals from the disease versus its effectiveness in reducing transmission from vaccinated individuals [4]. Nevertheless, even with a detailed understanding of trial dynamics, phase I and II statistics often lack the necessary precision. On the other hand, phase III trials involve a large number of participants, enabling more accurate estimations at the cost of insufficient detailed information to discern different types of efficacy and evaluate indirect effects. In this regard, leveraging digital contact tracing [20] along with national health service medical records can be crucial throughout various stages of a pandemic crisis. These measures aid in early detection, outbreak isolation [21], calibration of pharmaceutical and non-pharmaceutical interventions [13, 22], phase III efficacy estimation for rapidly developed vaccines, and phase IV evaluation of side effects once distributed among the population [18]. Nonetheless, we have demonstrated that collecting more reliable but less granular data, such as the fraction of infectious individuals in a population, would significantly improve the interpretation of medical trial results.

Indeed, we have illustrated how even a rudimentary understanding of epidemic data, such as estimates of the fraction of infectious individuals over time, can help rectify efficacy estimates from measured effectiveness during phase III (and potentially phase IV). Of course, more detailed data can provide insights into the impact of contact heterogeneity [22–24] on epidemic dynamics, vaccine efficacy, and herd immunity thresholds. Digital tracing data, in particular, would be invaluable for considering variations in contact patterns influenced by human behaviour. For instance, vaccinated individuals may modify their habits if they believe the vaccine offers protection. Therefore, without detailed contact information, these behavioural changes could introduce systematic biases in efficacy trials [10]. Overall, understanding and quantifying the effects of human behaviour are pivotal for effective policies regarding non-pharmaceutical interventions and vaccination strategies [25, 26].

Conclusions

Many factors impact the efficacy of a vaccine: from population-specific genetic characteristics to partial immunity acquired from previous infections, or even the development of variants during the epidemic: something that, given the duration of the still ongoing pandemic period, has occurred for COVID19. However, our study concentrates on the systematic underestimation of vaccine efficacy ϵ by the estimated vaccine effectiveness η in large cohort studies due to the presence of a high number of infectious individuals in the population. Since vaccines have never been produced, tested and experimented in such exceptional circumstances as the ones that recently occurred during COVID19, such an issue has not been fully addressed before.

For the sake of simplicity, we have employed a classical epidemiological model with realistic parameters to understand the order of magnitude of the systematic error in efficacy estimates; however, most models of epidemics do not differentiate between infection and disease, while there are cases where that the relation between the biological efficacy of the vaccine and its efficacy as measured by clinical trials is complex and multi-factorial [4, 27, 28]. Since disease (i.e. observable traits) is what drives behaviour, this is an issue that should be pursued further when modelling epidemics.

In the case of COVID19, age is a key factor with respect to the incidence of severe cases and/or mortality; in fact, efficacy estimates in phase III studies consider the effects of age [16, 29]. In such a case, we will have that effectiveness will be a larger underestimate of efficacy in the subgroups where the latter is lower. Moreover, since we have experienced that the COVID19 vaccine efficacy is time dependent and depends on the number of the doses [30], longitudinal studies should be planned in advance to detail the history-dependence of vaccine efficacy.

Finally, we notice that in the vaccine trials that have occurred, it has been observed that antibodies in the vaccinated individuals take time to develop [16, 29]; thus, if also the efficacy of a vaccine in a single individual grows with time, an extra bias could be introduced in efficacy and effectiveness measurements. In particular, if the trial occurs when the number of infectious is growing, the protection is low at the beginning of the trial, when the probability of getting infected is lower; on the contrary, if the epidemic is decreasing, the vaccine protects less at the beginning, i.e. just when the probability of getting infected is higher. Thus, for two trials—one before the epidemic peak, the other after—with identical time spans and attack rates, we expect a lower estimate of the vaccine efficacy (i.e. a larger systematic error) for the trial in the decreasing phase.

Supporting information

S1 Graphical abstract.

https://doi.org/10.1371/journal.pone.0290652.s001

(TIF)

Acknowledgments

AS thanks dr. Sandro Bonfigli for his valuable advice on EU and national health service issues, and dr. Emanuele Perugini for his precious advice on data sources for vaccine efficacy. Authors acknowledge the useful comments of referees on the Qeuios preprint server.

References

1. Shah A, Marks PW, Hahn SM. Unwavering Regulatory Safeguards for COVID-19 Vaccines. JAMA. 2020;324(10):931. pmid:32766736
- View Article
- PubMed/NCBI
- Google Scholar
2. Collins FS, Stoffels P. Accelerating COVID-19 Therapeutic Interventions and Vaccines (ACTIV): An Unprecedented Partnership for Unprecedented Times. JAMA. 2020;323(24):2455. pmid:32421150
- View Article
- PubMed/NCBI
- Google Scholar
3. WHO. Draft landscape and tracker of COVID-19 candidate vaccines; 2021. Available from: https://www.who.int/publications/m/item/draft-landscape-of-covid-19-candidate-vaccines.
4. Halloran ME, Longini IM, Struchiner CJ. Design and analysis of vaccine studies. Statistics for biology and health. New York: Springer; 2010.
5. Last JM, Association IE, editors. A dictionary of epidemiology. 4th ed. New York: Oxford University Press; 2001.
6. Revicki DA, Frank L. Pharmacoeconomic Evaluation in the Real World: Effectiveness Versus Efficacy Studies. PharmacoEconomics. 1999;15(5):423–434. pmid:10537960
- View Article
- PubMed/NCBI
- Google Scholar
7. Singal AG, Higgins PDR, Waljee AK. A Primer on Effectiveness and Efficacy Trials. Clinical and Translational Gastroenterology. 2014;5(1):e45. pmid:24384867
- View Article
- PubMed/NCBI
- Google Scholar
8. Teerawattananon Y, Anothaisintawee T, Pheerapanyawaranun C, Botwright S, Akksilp K, Sirichumroonwit N, et al. A systematic review of methodological approaches for evaluating real-world effectiveness of COVID-19 vaccines: Advising resource-constrained settings. PLOS ONE. 2022;17(1):e0261930. pmid:35015761
- View Article
- PubMed/NCBI
- Google Scholar
9. Diekmann O, Heesterbeek H, Britton T. Mathematical tools for understanding infectious diseases dynamics. Princeton series in theoretical and computational biology. Princeton: Princeton University Press; 2013. Available from: https://press.princeton.edu/books/hardcover/9780691155395/mathematical-tools-for-understanding-infectious-disease-dynamics.
10. Halloran ME, Longini IM, Haber MJ, Struchiner CJ, Brunet RC. Exposure efficacy and change in contact rates in evaluating prophylactic HIV vaccines in the field. Statistics in Medicine. 1994;13(4):357–377. pmid:8177986
- View Article
- PubMed/NCBI
- Google Scholar
11. Rossi D, Romano E, Guyennon N, Rainaldi M, Ghergo S, Mecali A, et al. The present state of Lake Bracciano: hope and despair. Rendiconti Lincei Scienze Fisiche e Naturali. 2018. https://doi.org/10.1007/s12210-018-0733-4
- View Article
- Google Scholar
12. Halloran ME, Struchiner CJ, Longini J Ira M. Study Designs for Evaluating Different Efficacy and Effectiveness Aspects of Vaccines. American Journal of Epidemiology. 1997;146(10):789–803. pmid:9384199
- View Article
- PubMed/NCBI
- Google Scholar
13. Scala A. The mathematics of multiple lockdowns. Scientific Reports. 2021;11(1):8078. pmid:33850217
- View Article
- PubMed/NCBI
- Google Scholar
14. Stanley HE. Introduction to phase transitions and critical phenomena. The International series of monographs on physics. Oxford: Clarendon Press; 1971.
15. Alimohamadi Y, Taghdir M, Sepandi M. Estimate of the Basic Reproduction Number for COVID-19: A Systematic Review and Meta-analysis. Journal of Preventive Medicine and Public Health. 2020;53(3):151–157. pmid:32498136
- View Article
- PubMed/NCBI
- Google Scholar
16. Polack FP, Thomas SJ, Kitchin N, Absalon J, Gurtman A, Lockhart S, et al. Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine. New England Journal of Medicine. 2020;383(27):2603–2615. pmid:33301246
- View Article
- PubMed/NCBI
- Google Scholar
17. Salje H, Kiem CT, Lefrancq N, Courtejoie N, Bosetti P, Paireau J, et al. Estimating the burden of SARS-CoV-2 in France. Science. 2020;369(6500):208–211. pmid:32404476
- View Article
- PubMed/NCBI
- Google Scholar
18. Forman R, Shah S, Jeurissen P, Jit M, Mossialos E. COVID-19 vaccine challenges: What have we learned so far and what remains to be done? Health Policy. 2021; p. S0168851021000853. https://doi.org/10.1016/j.healthpol.2021.03.013
- View Article
- Google Scholar
19. Ross R. An application of the theory of probabilities to the study of a priori pathometry.—Part I. Proceedings of the Royal Society of London Series A, Containing Papers of a Mathematical and Physical Character. 1916;92(638):204–230. https://doi.org/10.1098/rspa.1916.0007
- View Article
- Google Scholar
20. Moreno López JA, Arregui García B, Bentkowski P, Bioglio L, Pinotti F, Boëlle PY, et al. Anatomy of digital contact tracing: Role of age, transmission setting, adoption, and case detection. Science Advances. 2021;7(15):eabd8750. pmid:33712416
- View Article
- PubMed/NCBI
- Google Scholar
21. Kogan NE, Clemente L, Liautaud P, Kaashoek J, Link NB, Nguyen AT, et al. An early warning approach to monitor COVID-19 activity with multiple digital traces in near real time. Science Advances. 2021;7(10):eabd6989. pmid:33674304
- View Article
- PubMed/NCBI
- Google Scholar
22. Scala A, Flori A, Spelta A, Brugnoli E, Cinelli M, Quattrociocchi W, et al. Time, space and social interactions: exit mechanisms for the Covid-19 epidemics. Scientific Reports. 2020;10(1):13764. pmid:32792591
- View Article
- PubMed/NCBI
- Google Scholar
23. Mistry D, Litvinova M, Pastore y Piontti A, Chinazzi M, Fumanelli L, Gomes MFC, et al. Inferring high-resolution human mixing patterns for disease modeling. Nature Communications. 2021;12(1):323. pmid:33436609
- View Article
- PubMed/NCBI
- Google Scholar
24. Sun K, Wang W, Gao L, Wang Y, Luo K, Ren L, et al. Transmission heterogeneities, kinetics, and controllability of SARS-CoV-2. Science. 2021;371(6526):eabe2424. pmid:33234698
- View Article
- PubMed/NCBI
- Google Scholar
25. Read JM, Eames KTD, Edmunds WJ. Dynamic social networks and the implications for the spread of infectious disease. Journal of The Royal Society Interface. 2008;5(26):1001–1007. pmid:18319209
- View Article
- PubMed/NCBI
- Google Scholar
26. Funk S, Salathé M, Jansen VAA. Modelling the influence of human behaviour on the spread of infectious diseases: a review. Journal of The Royal Society Interface. 2010;7(50):1247–1256. pmid:20504800
- View Article
- PubMed/NCBI
- Google Scholar
27. Struchiner CJ, Halloran ME, Brunet RC, Ribeiro JMC, Massad E. Malaria vaccines: lessons from field trials. Cadernos de Saude Publica. 1994;10(suppl 2):S310–S326. pmid:15042221
- View Article
- PubMed/NCBI
- Google Scholar
28. Breslow NE, Storer BE. GENERAL RELATIVE RISK FUNCTIONS FOR CASE-CONTROL STUDIES. American Journal of Epidemiology. 1985;122(1):149–162. pmid:4014192
- View Article
- PubMed/NCBI
- Google Scholar
29. Voysey M, Clemens SAC, Madhi SA, Weckx LY, Folegatti PM, Aley PK, et al. Safety and efficacy of the ChAdOx1 nCoV-19 vaccine (AZD1222) against SARS-CoV-2: an interim analysis of four randomised controlled trials in Brazil, South Africa, and the UK. The Lancet. 2021;397(10269):99–111. pmid:33306989
- View Article
- PubMed/NCBI
- Google Scholar
30. Müller L, Andrée M, Moskorz W, Drexler I, Walotka L, Grothmann R, et al. Age-dependent Immune Response to the Biontech/Pfizer BNT162b2 Coronavirus Disease 2019 Vaccination. Clinical Infectious Diseases. 2021;73(11):2065–2072. pmid:33906236
- View Article
- PubMed/NCBI
- Google Scholar

[ref1] 1. Shah A, Marks PW, Hahn SM. Unwavering Regulatory Safeguards for COVID-19 Vaccines. JAMA. 2020;324(10):931. pmid:32766736
View Article
PubMed/NCBI
Google Scholar

[2] View Article

[3] PubMed/NCBI

[4] Google Scholar

[ref2] 2. Collins FS, Stoffels P. Accelerating COVID-19 Therapeutic Interventions and Vaccines (ACTIV): An Unprecedented Partnership for Unprecedented Times. JAMA. 2020;323(24):2455. pmid:32421150
View Article
PubMed/NCBI
Google Scholar

[6] View Article

[7] PubMed/NCBI

[8] Google Scholar

[ref3] 3. WHO. Draft landscape and tracker of COVID-19 candidate vaccines; 2021. Available from: https://www.who.int/publications/m/item/draft-landscape-of-covid-19-candidate-vaccines.

[ref4] 4. Halloran ME, Longini IM, Struchiner CJ. Design and analysis of vaccine studies. Statistics for biology and health. New York: Springer; 2010.

[ref5] 5. Last JM, Association IE, editors. A dictionary of epidemiology. 4th ed. New York: Oxford University Press; 2001.

[ref6] 6. Revicki DA, Frank L. Pharmacoeconomic Evaluation in the Real World: Effectiveness Versus Efficacy Studies. PharmacoEconomics. 1999;15(5):423–434. pmid:10537960
View Article
PubMed/NCBI
Google Scholar

[13] View Article

[14] PubMed/NCBI

[15] Google Scholar

[ref7] 7. Singal AG, Higgins PDR, Waljee AK. A Primer on Effectiveness and Efficacy Trials. Clinical and Translational Gastroenterology. 2014;5(1):e45. pmid:24384867
View Article
PubMed/NCBI
Google Scholar

[17] View Article

[18] PubMed/NCBI

[19] Google Scholar

[ref8] 8. Teerawattananon Y, Anothaisintawee T, Pheerapanyawaranun C, Botwright S, Akksilp K, Sirichumroonwit N, et al. A systematic review of methodological approaches for evaluating real-world effectiveness of COVID-19 vaccines: Advising resource-constrained settings. PLOS ONE. 2022;17(1):e0261930. pmid:35015761
View Article
PubMed/NCBI
Google Scholar

[21] View Article

[22] PubMed/NCBI

[23] Google Scholar

[ref9] 9. Diekmann O, Heesterbeek H, Britton T. Mathematical tools for understanding infectious diseases dynamics. Princeton series in theoretical and computational biology. Princeton: Princeton University Press; 2013. Available from: https://press.princeton.edu/books/hardcover/9780691155395/mathematical-tools-for-understanding-infectious-disease-dynamics.

[ref10] 10. Halloran ME, Longini IM, Haber MJ, Struchiner CJ, Brunet RC. Exposure efficacy and change in contact rates in evaluating prophylactic HIV vaccines in the field. Statistics in Medicine. 1994;13(4):357–377. pmid:8177986
View Article
PubMed/NCBI
Google Scholar

[26] View Article

[27] PubMed/NCBI

[28] Google Scholar

[ref11] 11. Rossi D, Romano E, Guyennon N, Rainaldi M, Ghergo S, Mecali A, et al. The present state of Lake Bracciano: hope and despair. Rendiconti Lincei Scienze Fisiche e Naturali. 2018. https://doi.org/10.1007/s12210-018-0733-4
View Article
Google Scholar

[30] View Article

[31] Google Scholar

[ref12] 12. Halloran ME, Struchiner CJ, Longini J Ira M. Study Designs for Evaluating Different Efficacy and Effectiveness Aspects of Vaccines. American Journal of Epidemiology. 1997;146(10):789–803. pmid:9384199
View Article
PubMed/NCBI
Google Scholar

[33] View Article

[34] PubMed/NCBI

[35] Google Scholar

[ref13] 13. Scala A. The mathematics of multiple lockdowns. Scientific Reports. 2021;11(1):8078. pmid:33850217
View Article
PubMed/NCBI
Google Scholar

[37] View Article

[38] PubMed/NCBI

[39] Google Scholar

[ref14] 14. Stanley HE. Introduction to phase transitions and critical phenomena. The International series of monographs on physics. Oxford: Clarendon Press; 1971.

[ref15] 15. Alimohamadi Y, Taghdir M, Sepandi M. Estimate of the Basic Reproduction Number for COVID-19: A Systematic Review and Meta-analysis. Journal of Preventive Medicine and Public Health. 2020;53(3):151–157. pmid:32498136
View Article
PubMed/NCBI
Google Scholar

[42] View Article

[43] PubMed/NCBI

[44] Google Scholar

[ref16] 16. Polack FP, Thomas SJ, Kitchin N, Absalon J, Gurtman A, Lockhart S, et al. Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine. New England Journal of Medicine. 2020;383(27):2603–2615. pmid:33301246
View Article
PubMed/NCBI
Google Scholar

[46] View Article

[47] PubMed/NCBI

[48] Google Scholar

[ref17] 17. Salje H, Kiem CT, Lefrancq N, Courtejoie N, Bosetti P, Paireau J, et al. Estimating the burden of SARS-CoV-2 in France. Science. 2020;369(6500):208–211. pmid:32404476
View Article
PubMed/NCBI
Google Scholar

[50] View Article

[51] PubMed/NCBI

[52] Google Scholar

[ref18] 18. Forman R, Shah S, Jeurissen P, Jit M, Mossialos E. COVID-19 vaccine challenges: What have we learned so far and what remains to be done? Health Policy. 2021; p. S0168851021000853. https://doi.org/10.1016/j.healthpol.2021.03.013
View Article
Google Scholar

[54] View Article

[55] Google Scholar

[ref19] 19. Ross R. An application of the theory of probabilities to the study of a priori pathometry.—Part I. Proceedings of the Royal Society of London Series A, Containing Papers of a Mathematical and Physical Character. 1916;92(638):204–230. https://doi.org/10.1098/rspa.1916.0007
View Article
Google Scholar

[57] View Article

[58] Google Scholar

[ref20] 20. Moreno López JA, Arregui García B, Bentkowski P, Bioglio L, Pinotti F, Boëlle PY, et al. Anatomy of digital contact tracing: Role of age, transmission setting, adoption, and case detection. Science Advances. 2021;7(15):eabd8750. pmid:33712416
View Article
PubMed/NCBI
Google Scholar

[60] View Article

[61] PubMed/NCBI

[62] Google Scholar

[ref21] 21. Kogan NE, Clemente L, Liautaud P, Kaashoek J, Link NB, Nguyen AT, et al. An early warning approach to monitor COVID-19 activity with multiple digital traces in near real time. Science Advances. 2021;7(10):eabd6989. pmid:33674304
View Article
PubMed/NCBI
Google Scholar

[64] View Article

[65] PubMed/NCBI

[66] Google Scholar

[ref22] 22. Scala A, Flori A, Spelta A, Brugnoli E, Cinelli M, Quattrociocchi W, et al. Time, space and social interactions: exit mechanisms for the Covid-19 epidemics. Scientific Reports. 2020;10(1):13764. pmid:32792591
View Article
PubMed/NCBI
Google Scholar

[68] View Article

[69] PubMed/NCBI

[70] Google Scholar

[ref23] 23. Mistry D, Litvinova M, Pastore y Piontti A, Chinazzi M, Fumanelli L, Gomes MFC, et al. Inferring high-resolution human mixing patterns for disease modeling. Nature Communications. 2021;12(1):323. pmid:33436609
View Article
PubMed/NCBI
Google Scholar

[72] View Article

[73] PubMed/NCBI

[74] Google Scholar

[ref24] 24. Sun K, Wang W, Gao L, Wang Y, Luo K, Ren L, et al. Transmission heterogeneities, kinetics, and controllability of SARS-CoV-2. Science. 2021;371(6526):eabe2424. pmid:33234698
View Article
PubMed/NCBI
Google Scholar

[76] View Article

[77] PubMed/NCBI

[78] Google Scholar

[ref25] 25. Read JM, Eames KTD, Edmunds WJ. Dynamic social networks and the implications for the spread of infectious disease. Journal of The Royal Society Interface. 2008;5(26):1001–1007. pmid:18319209
View Article
PubMed/NCBI
Google Scholar

[80] View Article

[81] PubMed/NCBI

[82] Google Scholar

[ref26] 26. Funk S, Salathé M, Jansen VAA. Modelling the influence of human behaviour on the spread of infectious diseases: a review. Journal of The Royal Society Interface. 2010;7(50):1247–1256. pmid:20504800
View Article
PubMed/NCBI
Google Scholar

[84] View Article

[85] PubMed/NCBI

[86] Google Scholar

[ref27] 27. Struchiner CJ, Halloran ME, Brunet RC, Ribeiro JMC, Massad E. Malaria vaccines: lessons from field trials. Cadernos de Saude Publica. 1994;10(suppl 2):S310–S326. pmid:15042221
View Article
PubMed/NCBI
Google Scholar

[88] View Article

[89] PubMed/NCBI

[90] Google Scholar

[ref28] 28. Breslow NE, Storer BE. GENERAL RELATIVE RISK FUNCTIONS FOR CASE-CONTROL STUDIES. American Journal of Epidemiology. 1985;122(1):149–162. pmid:4014192
View Article
PubMed/NCBI
Google Scholar

[92] View Article

[93] PubMed/NCBI

[94] Google Scholar

[ref29] 29. Voysey M, Clemens SAC, Madhi SA, Weckx LY, Folegatti PM, Aley PK, et al. Safety and efficacy of the ChAdOx1 nCoV-19 vaccine (AZD1222) against SARS-CoV-2: an interim analysis of four randomised controlled trials in Brazil, South Africa, and the UK. The Lancet. 2021;397(10269):99–111. pmid:33306989
View Article
PubMed/NCBI
Google Scholar

[96] View Article

[97] PubMed/NCBI

[98] Google Scholar

[ref30] 30. Müller L, Andrée M, Moskorz W, Drexler I, Walotka L, Grothmann R, et al. Age-dependent Immune Response to the Biontech/Pfizer BNT162b2 Coronavirus Disease 2019 Vaccination. Clinical Infectious Diseases. 2021;73(11):2065–2072. pmid:33906236
View Article
PubMed/NCBI
Google Scholar

[100] View Article

[101] PubMed/NCBI

[102] Google Scholar

Figures

Abstract

Introduction

Methods

Vaccine efficacy

SIR model

Stochastic estimates of the efficiency

Results

Discussion

Conclusions

Supporting information

S1 Graphical abstract.

Acknowledgments

References