Data sources and structure

We consider a scenario in which time-series of cumulative reported cases, deaths and vaccination doses are actively maintained and monitored during the epidemic. There are a number of challenges in using this type of data source to model the epidemic evolution and understanding hesitancy. Firstly, the cumulative confirmed cases do not necessarily represent the actual number of infections that have occurred in the population. Primarily this will be due to undetected asymptomatic cases [25,26], but could also be impacted to a substantially lesser degree by lower sensitivity or specificity in diagnostic test results. These undetected cases need to be accounted for since asymptomatic cases can still effectively transmit disease [27]. Second, the cumulative case recoveries often needs to be estimated as a proportion of cumulative cases that did not die and are not currently active. Finally, and most importantly for the study of vaccine hesitancy, the cumulative case and death data is often not stratified by vaccination status. While the vaccination status of deaths is recorded by hospitals [28], it is typically not available in large scale aggregated data available from CDC dashboards [29,30]. Therefore the distribution of vaccination status across cases and deaths also needs to be inferred when working with dashboard data.

Since this work is designed to evaluate the potential of our methods to identify hesitancy behaviour patterns, we rely on simulated data using a model with known hesitancy patterns. However, our modelling and simulated data processes are constructed in such a way that they are consistent with real epidemiological data repositories such as those provided by Johns Hopkins University [29] or Our World in Data [30] during the COVID-19 pandemic. For our simulation study, we have selected key population and epidemiological parameters that are qualitatively similar to real scenarios that occurred during the COVID-19 pandemic, however, our various hesitancy scenarios are all hypothetical and no conclusions are claimed in relation to vaccine hesitancy during COVID-19 vaccination roll-outs.

Model development

Here, we describe our stochastic epidemiological model that incorporates both the effects on NPIs and the effect of vaccination hesitancy. Our approach builds upon the model employed by Warne et al., [21] and Le et al., [22] to quantify community behaviour in relation to NPIs and potential strategies for NPI relaxation. Nomenclature is provided in Tables 1 and 2 for reference throughout theses sections. For full mathematical details of the models see S1 Appendix.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. List of compartments in the full SEIR model including the case reporting process of a two-dose vaccination program. A state is considered observable if it is available in a publicly available repository. States labelled with an asterisk “^*” indicate data used in reporting, but aggregated before public release.

https://doi.org/10.1371/journal.pone.0349499.t001

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Model parameters fixed for synthetic dataset generation. Parameters related to vaccination rates and hesitancy are estimated via Bayesian inference. All other parameter values are obtained from the literature [21,31].

https://doi.org/10.1371/journal.pone.0349499.t002

Our model is a stochastic susceptible-exposed-infectious-recovered (SEIR) compartmental model. In this framework, a population of size P is divided into four main epidemiological compartments: susceptible (S), exposed (E), infectious (I) and recovered (R). Since we cannot observe exactly the true number of active infectious individuals, these components are considered unobservable or latent states. We consider the introduction of a vaccination program including a single vaccine-brand with a two dose vaccination protocol (e.g., AstraZeneca or Pfizer only), that is, each compartment is further resolved by vaccination status: unvaccinated, vaccinated (1st dose), and fully vaccinated (2nd dose) (Fig 1).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Schematic of the epidemiological model including two-dose vaccination program.

States are marked as labelled boxes and arrows indicate state transitions. The model consists of three vaccinations stages, each with its own SEIR model with the reported case data arising through the transition from I_u to (resp. I₁ to and I₂ to ). The transmission is inhibited by a response function that depends on the aggregated observed reported case data. Similarly, vaccination is promoted through the hesitancy effect function that depends on the aggregated observed reported case data.

https://doi.org/10.1371/journal.pone.0349499.g001

Since the population counts within each of these compartments are not directly observable in reality, these states are treated as unobservable latent states and we include an observation process based on diagnostic testing in the infectious population. Within the same vaccination status i, infectious individuals (I_i) will be identified as active confirmed cases () at rate . These active confirmed cases will transition to reported deaths () or recoveries () at rates and respectively. If we accumulate these observable case numbers over vaccination status, then we obtain three observables: the total confirmed active cases , total case deaths, , total case recoveries, , total vaccinations (1st dose), , and total fully vaccinated . These observables correspond to reported data available from online dashboards and repositories during the COVID-19 pandemic [29,30]. It should be noted that most data sources only reliably record the cumulative case numbers, , and reported deaths, D^*. In addition, the reported C^* will be an underestimate of the true number of cumulative cases since infectious individuals can also recover without being reported as a confirmed case. In our model this transition occurs with rate .

We assume vaccination only affects the disease transmission and mortality rate. At each vaccination level , modifiers and are applied to the baseline transmission rates and mortality rate , respectively. For example, at vaccination level i the effective mortality rate is . We set and assume and . Estimates for these modifier parameters would typically be obtained from the literature resulting from vaccination clinical trials [28,31–33]. For the purposes of our simulation study, we derived rough estimates based on literature on the Oxford-AstraZeneca vaccine (ChAdOx1) in relation to the B.1.617.2 (Delta) variant of SARS-CoV-2 virus [31]. There are also vaccination rate parameters, and . Here, represents the rate at which individuals get vaccinated in the absence of any vaccine hesitancy, and is the rate at which individuals who received their first dose days ago will proceed to second dose as per protocol, this is implemented using a delay term (See S1 Appendix). We assume that those currently infected, that is exposed and infectious individuals, do not get vaccinated until after recovery. This simplifies the model so that the only possible transitions between vaccination levels are for the susceptible and recovered compartments, S_i, R_i, and .

Given the large well-mixed populations that we consider here (Fig 2), it is true that deterministic models may provide good approximations for the mean, however there are a number of key reasons we focus on the stochastic setting in this work. Firstly, even though the observable states are large, this does not mean that extinction events or very small populations cannot occur within any single component; ignoring this possibility can have a non-negligible effect on the system dynamics due to the two non-linear feedback mechanisms. Secondly, even when the simulation error is small, this can have an unpredictable effect on parameter inference and uncertainty quantification [34,35]. Finally, we wish to provide a computational framework that is widely applicable to other settings that include smaller populations [21]. While working with a stochastic model comes at a computational cost, we mitigate this computational burden using efficient approximate stochastic simulation and Bayesian inference scheme.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Comparison of the effect of an AstraZeneca-like vaccination program under different configurations of response function and vaccination inclusion (assuming no vaccine hesitancy).

Simulations are initialised based on the UK data on 1st September 2020 with , . (A) and (C): corresponds to ongoing small lockdowns, (B) and (D): corresponds to complete easing of restrictions after T ≥ 92 (roughly the 2nd of December 2020). A single stochastic realisation is shown in (A)–(D) using an approximate stochastic simulation scheme (See S1 Appendix).

https://doi.org/10.1371/journal.pone.0349499.g002

In our model, the virus transmission rate and the vaccination rate are modulated by two feedback mechanisms representing how a population changes their behaviour in response to information related to the observable states and . The first is a negative feedback response function that affects the transmission rate and represents the implementation of voluntary or mandated NPIs including the compliance thereof (See S1 Appendix for details). Such mechanisms have been explored for a variety of infectious disease to identify media influences [21,22,36,37]. A variety of forms for this response function can be implemented [21], here we employ an NPI implementation strategy based on a trigger threshold,

(1)

where T_d > 0 is the point in time that NPIs cease and w_A is a weight parameter setting the number of reported active cases at which NPI efficacy reaches 50%. Note that for t > T_d the . See S1 Appendix for additional details.

Fig 2 demonstrates the effect that vaccination has on the virus spread and severity using example stochastic realisations from our model parameterised by Fig 2 under different scenarios of response behaviour with the assumption no hesitancy behaviour (that is, perfect vaccination rate of for where T_v is the start time of the vaccination roll-out). Fig 2(A) shows a continued lockdown using Eq. (1) with and no vaccination roll-out . Fig 2(B) shows a similar scenario, however with NPIs ceasing at T_d = 3 (months). Comparing Fig 2(A) and 2(B) demonstrates the effect of ceasing NPIs in the absence of any effective vaccine, that is, an order of magnitude more reported cases and deaths. Fig 2(C) and 2(D) show the same lockdown scenarios as Fig 2(A) and 2(B), however, introducing a vaccination roll-out commencing at T_v = 3 (months). We observe, as expected, a reduction in the total case deaths due to vaccination (Compare Fig 2(A) with 2(C) and Fig 2(B) with 2(D)). We also note a chance of elimination occurring when restrictions continue along with the vaccination program (Fig 2(C)), such elimination behaviours can only be captured using a stochastic model.

The second feedback loop relates to the vaccine hesitancy. We denote as the hesitancy effect function resulting in an effective vaccination rate of . We assume that vaccine hesitancy behaviour only affects the probability of an individual receiving the first dose with every individual that gets vaccinated continuing to obtain a second dose to be “fully vaccinated”. We note that this assumption may not always be appropriate, especially for diseases requiring boosters or adverse reactions to the first dose reducing the likelihood of an individual returning for the second dose (see Discussion). Just as with the response function, the hesitancy effect function only depends on the observable states and time. We treat as an increasing function with corresponding to a situation where individuals refuse vaccination entirely and leads to no hesitancy effect with the maximum rate achieved. This function is intended to model the way in which reported case and vaccination data may increase the probability of an individual seeking vaccination.

We can construct a hesitancy effect function to capture the complacency and confidence components of the 3C model of vaccine hesitancy [4]. We do not consider the convenience component due to the well-mixed assumption (See Discussion). If we consider a population that is only affected by complacency, then it is reasonable to assume the incidence of a disease in a community will tend to increase the likelihood of an individual to seek vaccination [16,18] as the perceived risk increases to overcome complacency. This complacency pattern could be modelled with the function,

(2)

where are weights reflecting the influence of data reports on population behaviour and n_v > 0 is a slope parameter that governs the sensitivity of the behaviour change to increased incidence, and T_v is that time the vaccine roll-out starts with for t < T_v. The relative importance of case numbers and deaths in influencing an individuals decision to get vaccinated can be assessed by considering the case where , that is each new death has as much influence as new cases. Of course, motivation could be completely dominated by cases (resp. deaths), in which case w_D = 0 (resp. w_C = 0). In practice, due to the strong correlation between total confirmed cases and deaths, it is perfectly reasonable to set .

Another cause of vaccine hesitancy behaviour are vaccine safety concerns, related to confidence in the 3C model [4]. Such safety concerns were prevalent in the COVID-19 pandemic due to rapid development of vaccines [38] and some widely publicised, though rare, side effects [19,20]. If only vaccination safety is a concern, then critical mass of fully vaccinated individuals may be necessary to alleviate these concerns. This can be reflected in the function,

(3)

with weight parameter w_V > 0. Note that in this case, at time T_v some initial non-zero vaccinated population must be present, otherwise the vaccinated population will remain at . The initial population could represent participants in relevant clinical trials, or an initial vaccination mandate for particular occupations (e.g., healthcare workers). In reality, other factors beyond vaccine safety could affect confidence especially in situations where the vaccine reduces disease severity without impacting transmission. In this setting, Eq. (3) could be extended to enable continued increase in reported cases to reduce confidence, which would work against the increasing case numbers in the complacency function Eq (2).

Finally, we can consider a population hesitancy behaviour that is due to safety concerns and complacency,

(4)

Once again the weights provide a measure of the relative influence each new case, death or vaccination has on the probability of an individual getting vaccinated. In reality, these parameters will be unknown, however if standard COVID-19 case data can provide insight into the values of these parameters [21], then we can start to assess the trends in hesitancy behaviours for a given population of interest.

Simulation study

Given that reported data are aggregated, it is unclear if the data are informative enough to identify what type of hesitancy effect is occurring in reality. For example, three different scenarios are shown in Fig 3 corresponding to no hesitancy (Fig 3(A)), complacency only (Fig 3(B)), and complacency and vaccine safety concerns (Fig 3(C)). While each evolution is different qualitatively, it is completely unclear if the scenarios including different hesitancy effects can be distinguished from a scenario with no hesitancy effect but slower vaccination rate. For example, such is the difference in the hesitancy behaviour between Fig 3(C) and the no hesitancy case of Fig 3(A), yet the differences in the dynamics are subtle. Here Fig 3(C) looks like Fig 3(A) with a slower vaccination rate, however, in reality both have the same rate and the effect is entirely due to hesitancy.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Comparison of different hesitancy effect functions show the dynamics of the observable states.

The scenarios are: (A) no hesitancy behaviour with ; (B) hesitancy effect driven by complacency only with defined in Eq. (2); and (C) hesitancy effect driven by mild safety concerns and some complacency only with defined in Eq. (4). Simulations are initialised based on the UK data on 1st September 2020 with , , and model parameters are given in Table 2 with , n_v = 4, , , and .

https://doi.org/10.1371/journal.pone.0349499.g003

The behaviour observed in Fig 3 provides motivation to explore the informativity of aggregated reported COVID-19 data as provided by online repositories synthetically for the purpose of analysing vaccine hesitancy behaviour. To explore this, we consider a simulation study and develop Bayesian analysis techniques.

We generate multiple simulated datasets under three different vaccine uptake behaviours. Specifically, we consider: complacency-based hesitancy (w_V = 0, w_C > 0, and w_D > 0), complacency and vaccine safety (w_V > 0, w_C > 0, and w_D > 0) hesitancy, and no hesitancy (). In each of the simulation settings we vary the weights and the maximum vaccination rate, the remaining parameters are fixed based on the literature [21,28,31–33] as listed in Table 2.

Bayesian analysis

Given a simulated dataset, , we infer the parameters related to the vaccination uptake through sampling the Bayesian posterior distribution with density,

where is the likelihood function and is the prior probability density. Due to the stochastic nature of the model and the large number of latent states, the likelihood function is intractable, and we rely upon approximate Bayesian computation (ABC) [39,40] to sample the approximate posterior,

(5)

where is a simulated dataset generated from the model with given parameter values, , is a discrepancy metric between the data and simulations, and is a target discrepancy threshold. For this work, our discrepancy metric between two datasets is the Frobenius norm of the matrix of differences between counts in each dataset (S2 Appendix). To achieve accurate inferences we apply an adaptive sequential Monte Carlo sampler for ABC (SMC-ABC) [41] that produces N samples with as small a discrepancy as practically possible. We repeat this inference process for all M synthetic datasets, , that span the different hesitancy scenarios. Thereby arriving at M sets of posterior samples . The structure of the resulting posterior samples can be used to investigate if there is information about vaccine hesitancy available in COVID-19 case data and vaccination counts. In all cases the priors are , , , , and .

Model fit is assessed through sampling the within-sample posterior predictive distribution with density,

(6)

If the model fits well, then we expect the data to lie within the 95% Credible Intervals (95% CrI). Since the models we use to generate synthetic data are all special cases of the full model that is based on (4) and (5), there is no model misspecification error to account for in the inference. However, the posterior predictive check is useful to assess the accuracy of the SMC-ABC sampler with respect to the discrepancy threshold. In practice, model misspecification is important to consider in real applications and we discuss this in the Discussion section.

Identifiability analysis

In the setting of deterministic models, a parameter is defined as structurally identifiable if the mapping from parameter value to a model solution curve is an invertible function. Various methods to analyse structural identifiability for deterministic dynamical systems are available in the literature [42–45]. In contrast, very few methods are available for structural identifiability for stochastic models [24,46]. For simplicity, we derive an equivalent deterministic ordinary differential equation (ODE) approximation (S1 Appendix) to our stochastic model and apply generating series approaches to investigate identifiability [45]. This is appropriate in this setting as the ODE can be considered as an approximation of the mean of the stochastic process as the variance is comparatively negligible due to the large populations in each compartment [24].

In addition to structural identifiability, Bayesian posterior samples provide a way to explore practical identifiability. While frequentist approaches exist for exploring practical identifiability, such as profile likelihoods [47–49], they are challenging to apply for models with intractable likelihood functions [50]. Therefore we adopt a Bayesian approach where practical identifiability is primarily explored via the marginal posterior distributions typically using Markov chain Monte Carlo [24,51] and posterior predictive sampling. Since we are sampling from the joint posterior distribution we automatically have access to samples from the marginal posterior distributions. When marginals themselves demonstrate practical non-identifiability, exploring correlation structures can provide additional insight. In this work, we visually inspect the bivariate marginal structures for each simulation case and Spearman’s rank correlation between pairs of parameters.

Simulation scenarios and model calibration

Synthetic datasets were constructed using the model under each vaccination hesitancy behaviour scenario considered in the Methods section. Example synthetic datasets for these three cases are shown in Fig 4.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. Example synthetic datasets for the three hesitancy scenarios:

(A) no hesitancy, but smaller maximum vaccination rate than in (B) and (C); (B) complacency only with w_C > 0, w_D > 0 and w_V = 0; and (C) complacency and vaccine safety concerns with w_C > 0, w_D > 0 and w_V > 0.

https://doi.org/10.1371/journal.pone.0349499.g004

For each dataset, , we apply adaptive SMC-ABC (S2 Appendix) targeting the joint posterior . The result is N = 1000 samples from the approximate posterior for each synthetic dataset. These samples are used for Monte Carlo integration of posterior probability densities, practical identifiability analysis, and visualisation of parameter correlation structures to investigate the use of COVID-19 case data for identification of vaccine hesitancy patterns.

The model fitness is evaluated for each synthetic dataset using within-sample posterior predictive checks. Since the model used for each synthetic dataset is nested within our full model we do not need to account for model misspecification. See Discussion for considerations when misspecification is relevant. The purpose of the posterior predictive check is to ensure the sampler has adapted to a sufficiently small discrepancy threshold for inference. Fig 5 provides example posterior predictive simulations and demonstrates typical model fits obtained from the Bayesian calibration using equivalent convergence criteria for the posterior sampler.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 5. Example posterior predictive simulations (shaded regions) demonstrating model fit for different synthetic datasets (black dashed lines).

(A) no hesitancy (true parameters ); (B) complacency only (true parameters ); and (C) Complacency and vaccine safety concerns (true parameters ).

https://doi.org/10.1371/journal.pone.0349499.g005

Identifiability of hesitancy function parameters

We use the GenSSI 2.0 toolkit [43,45] to analyse the structural identifiability of the parameters related to vaccination uptake . This approach is applied to a deterministic version of the model. We consider the case when T_v = T_d, that is, NPI based restrictions are lifted at the same time as the vaccine roll-out, and the case when , that is, NPI restrictions continue throughout the vaccination roll-out. In both cases we observe only the total reported case numbers, deaths, and vaccinations.

In both cases, all of the vaccine uptake parameters are shown to be locally structurally identifiable. This means that model parameters can be uniquely determined within a subset of parameter space. While this structural identifiability analysis is not directly applicable to the stochastic model, for very large case numbers, the stochastic effects are negligible. Therefore, we conclude that the observation process, if it was continuous, is informative enough to investigate hesitancy behaviours.

In addition, since our real observation process only records daily counts, we still need to explore practical identifiability as the data are not continuous noise-free observations. The posterior marginal distributions demonstrate practical non-identifiability. This can be observed in the example marginal posterior density plots (Fig 6). Regardless of the synthetic data scenario, the hesitancy function parameters and w_V shows substantial uncertainty. The only parameter that is well identified is the maximum vaccination rate . While the shape of the marginals (particularly the skewness) provide some hints toward recovering the true hesitancy scenario for the respective synthetic data, the differences between the scenarios are subtle.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 6. Example marginal posterior densities obtained using synthetic datasets for the three hesitancy scenarios: (A) no hesitancy; (B) complacency only; (C) complacency and vaccine safety concerns.

True parameter values are indicated (red dashed lines). Substantial uncertainties in the hesitancy function parameters n_v, w_C, w_D, and w_V indicate practical unidentifiability problems. Cases without a dashed red line indicate cases when the true parameter corresponds to: (A) ; and (B) .

https://doi.org/10.1371/journal.pone.0349499.g006

Hesitancy inferred through dependency structures

The marginal posterior distributions indicate that key parameters related to vaccination rates and hesitancy behaviours are practically non-identifiable using COVID-19 case data. However, practical non-identifiability does not preclude the study of the posterior structure to provide insight into the hesitancy behaviours. In particular, the joint posterior distributions contain information about the correlation structure between these parameters. The qualitative form of these correlation structures can be used to uncover the true hesitancy behaviour, albeit without the ability to quantify the actual parameter values reliably. Fig 7 show characteristic example bivariate plot matrices for each of the different vaccination hesitancy scenarios (See S3 Appendix for full bivariate distributions). Throughout, all statistical hypothesis tests are performed at the 0.01 significance level, and correlations refer to Spearman’s rank correlation.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 7. Univariate and bivariate marginal distributions obtained using synthetic datasets for the three hesitancy scenarios: (A) no hesitancy; (B) complacency only; and (C) complacency and vaccine safety concerns.

The univariate marginal posterior densities for the maximum vaccination rate, , is show with dashed red lines indicating the true values. The scatter plots show the bivariate posterior samples involving , from which the pairwise correlations are computed. Plots with significantly positive, significantly negative and statistically insignificant correlations are shown in red, blue and grey respectively.

https://doi.org/10.1371/journal.pone.0349499.g007

Firstly, by comparing Fig 7(A), that corresponds to synthetic data generated with no hesitancy, with Fig 7(B) and 7(C), that correspond to different hesitancy scenarios, we observe qualitative and quantitative differences in the bivariate posteriors. In the absence of hesitancy in the data, there is no evidence for correlation between and the negative log weights, or with the slope parameter n_v as all correlations are statistically insignificant (Fig 7(A)). In contrast, there are statistically significant negative correlations between and n_v, and statistically significant positive correlations between and negative log weights (Fig 7(B) and 7(C)). This demonstrates that correlation between the maximum vaccination rate with the negative log weight parameters , , and provide a strong indicator for the presence of hesitancy behaviours, despite univariate marginal parameter estimates being practically non-identifiable.

Second, comparing correlation structures for different cases of hesitancy can inform the components that are dominating behaviour. For example, Fig 7(B) shows the results for a synthetic dataset with only complacency behaviour. Here, the strength of the correlation between and is substantially stronger than that of with and . This indicates the vaccine counts have a much smaller effect on the vaccine uptake than the confirmed cases and deaths. By comparison, a scenario that includes both complacency and vaccine safety concerns (Fig 7(C)) has weaker correlation strengths that are on similar scales. The interpretation here is that increased cases, deaths and vaccine counts all increase the probability of individuals get vaccinated.

Finally, the correlations between the slope n_v parameters also provide some insight, but this information mainly complements the insights obtained through correlations with and the other parameters (S3 Appendix). These correlations seem to allow some identification of the dominating complacency drivers between total case numbers or death numbers, however, more work is required to explore this in more detail.

Summary

Through simulated synthetic dataset scenarios and Bayesian analysis, we demonstrate the ability to identify the presence of vaccine hesitancy at a population level. This is possible despite practical non-identifiability of individual parameters. We achieve this though inspection of the correlation structures in the joint posterior distribution for the parameters relevant to vaccine hesitancy. Further, once vaccine hesitancy behaviours have been identified, it is possible to isolate the specific factors that are driving the hesitancy behaviour. We explore the potential implications and limitations of these results in the Discussion section.