Subject recruitment

We conducted our survey in three counties: Canada, Japan, and the United States. Our multi-country, cross-national approach is designed primarily to assess the robustness and generalizability of our findings rather than to examine country-specific variations. Given this objective, we chose countries with varying attitudes toward the WHO, vaccine hesitancy levels, political relations with China, and COVID-19 infection rates at that time (in addition to practical considerations, including contextual familiarity, availability of high-quality survey participants, and financial constraints).

We recruited the respondents in late November 2020, after the first viable and effective vaccines were announced but before any were officially approved by a government. We recruited participants using Prolific, an online platform based in the United Kingdom where we set criteria and eligible individuals have the opportunity to participate if they meet criteria. Research demonstrates that Prolific recruits a more diverse and higher-quality participant pool than comparable online labor platforms (e.g., Amazon Mechanical Turk) and conventional online panels (e.g., Qualtrics) [40–42]. Specifically, we recruited 832 respondents from Canada (11/20/2020–11/24/2020) and 1,001 in the United States (11/23/2020–11/24/2020). For Japan, we used Yahoo! Crowdsourcing, a Japan-based online platform, to recruit 1,474 respondents (11/21/2020). These sample sizes are comparable to or exceed those commonly employed in standard conjoint experiments.

Unsurprisingly, our respondents were younger and more left/liberal-leaning and less right/conservative-leaning (see S1 Fig for the standardized differences in means between the sample and target population). While previous validation efforts show experiments using opt-in samples, such as from Prolific, consistently replicate the signs of treatment effect estimates carried out in random samples of the target populations [43,44], reweighting to match demographic moments of opt-in samples to the target populations’ can help bring magnitudes of the treatment effects more in line [45]. We turn to entropy balancing [46] to balance the country-specific samples so that the mean age and proportions of left/liberals, right/conservatives, and women match between the samples and the target datasets. Our reweighting procedure improves the overall balance of the data, though some imbalances remain, particularly with respect to education and gender in the Canadian and US samples (see S1 Fig). All the analyses below use these country-specific weights.

Experimental design

We use a conjoint design, an approach previously used to study vaccine preferences [10,13]. In our study, we ask respondents to evaluate a series of five pairs of hypothetical vaccines, each differing in specific attributes. All attributes and potential realizations are summarized in Table 1. All realizations were drawn from a uniform distribution, recognizing ignorance about what a future vaccine might be like.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Vaccine attributes and endorsements, possible realizations.

https://doi.org/10.1371/journal.pgph.0005410.t001

We divide the attributes into two blocks: vaccine features and endorsements. The vaccine features include duration of effectiveness, risks associated with mild and severe side effects, efficacy against severe cases, and the vaccine’s country of origin—China, Germany, the United Kingdom, or the United States. These attributes enable us both to replicate and validate our conjoint design, and to examine whether endorsement effects hold for vaccines of lower overall quality.

Our objective is to assess how the persuasive impact of WHO endorsements compares with endorsements issued by other influential global actors. Building on scholarship on cue-taking [6,29,30], we theorize that individuals evaluate vaccine endorsements not only in terms of technical expertise but also with regard to perceptions of neutrality and independence. Endorsements from institutions seen as neutral and technically competent may therefore be more influential than those viewed as politically embedded or contested. This logic is particularly relevant for the WHO, which holds a dual identity. It is widely recognized as scientific authority, yet simultaneously an intergovernmental body whose decisions are vulnerable to politicization, a dynamic that became especially salient during the COVID-19 pandemic [18].

Guided by this framework, we selected three comparator organizations that (1) were salient across Canada, Japan, and the United States in late 2020, (2) exemplify distinct institutional bases of legitimacy, and (3) plausibly compete with or complement WHO endorsements. Oxford University embodies academic independence, with its credibility further reinforced by its prominent role in COVID-19 vaccine development. The Gates Foundation symbolizes global philanthropy and policy entrepreneurship, reflecting authority rooted in resources and prominence in global health advocacy. The CDC represents national bureaucratic authority with long-standing technical competence in public health, though its proximity to domestic politics may complicate perception of independence. Together with the WHO, these organizations span the major non-commercial sources of health guidance available to global publics during the pandemic. Their contrasts also allow us to investigate whether WHO’s influence is reinforced, substituted, or attenuated when endorsements come from institutions grounded in different forms of legitimacy.

We restrict the design to this set of four endorsers in order to maintain cognitive tractability for respondents, while still enabling systematic variation in the number of co-endorsements. This structure allows us to evaluate whether additional signals produce additive, complementary, or diminishing effects on the persuasive impact of WHO’s endorsement.

Each vaccine profile in our study indicates whether it is endorsed by these actors, reflecting their diverse influences and potential impacts. We use a checkmark symbol for endorsement and a dash sign for non-endorsement. The latter is expected to be understood as “no information” about the vaccine from that entity, rather than as suggesting disapproval. However, the absence of endorsement can be informative in itself as it might raise questions about the quality of the vaccine.

Prior to the profiles of vaccines, we provided respondents with brief introductions to the four endorsers and asked for them to rate their feelings towards these entities using a 100-point scale thermometer before introducing the hypothetical vaccines (see S1 File for greater details). As part of our original experimental design, we also included a specific treatment in the WHO introduction to assess if perceptions of bias affected its endorsement impact. We created two versions of the WHO introduction. Half of the respondents read that “the WHO is the world’s most important international organization dedicated to all matters of global health,” whereas the other half see the same introduction with an additional note saying “[during] the early days of the current Coronavirus pandemic, the WHO was sharply criticized for its deference to China where the novel coronavirus originated.”

We decided to omit this aspect of the analysis as our manipulation check—the thermometer scores—revealed that this additional note on China did not significantly alter perceptions of the WHO in two of the three countries surveyed, contrary to our intentions; in one country, the effect magnitude was tiny. Therefore, in the subsequent analyses, we will not distinguish between the two versions of the WHO and pool all responses. We elaborate on this decision in S1 Appendix.

Outcome variable

Following the approach used by Kobayashi et al., we ask respondents to rate each vaccine in each pair using an ordinal scale to determine the expected timeframe for vaccine uptake [24]. This approach differs from the binary choice used in much previous research, offering two main advantages. First, our measure yields a richer set of data that goes beyond a simple yes/no binary. Second, the timing of vaccinations is crucial for public health outcomes, as achieving herd immunity necessitates widespread vaccination. Therefore, understanding the distribution of intended vaccination timings provides more insightful analysis as our simulations will show.

Respondents were given five options to express whether and when they would choose to get vaccinated if a vaccine were available at no cost. These options were: “Yes, within a month," “Yes, within 2-3 months,” “Yes, within 4-12 months,” “Yes, after a year,” and “No, never.” To simplify our analysis and the interpretation of results, we consolidated these responses into a three-level ordered variable: the “early” category for intentions within three months, “middle” for the 4-12 month timeframe, and “late" for wait times of a year or more, including “never.” This three-level ordinal variable offers a more detailed understanding of vaccination intentions. For the simulations of the vaccination shares, we will use all levels.

Additional variables

Early in the survey, we ask respondents to answer a series of questions following the general vaccine hesitancy literature [38], including how they feel about vaccine safety and necessity [39]. We asked about their degree of agreement or disagreement with the statements (seven levels), “Adults should get all recommended vaccines", “Children should at all recommend vaccines", and “General speaking, recommended vaccines are safe." For each survey country, we take the three answers, treat them as approximately linear, standardize each by its mean and standard deviation, and then take the average. This is our index of vaccine attitudes. For the U.S. and Canada samples, Cronbach’s α rounds to 0.97 for each; in Japan, we dropped the third item, which improves the α from 0.59 to 0.83. We used simple imputation to fill missing vaccine attitudes index observations for 0.8% of respondents (n=27). Further, we also collect standard demographic information, such as age, gender, education, and ideological orientation. Since these differ slightly by country (see below), we relegate details to S1 File.

Ethics statement

Research ethics approval for the experiment was obtained from the Research Ethics Committee of Graduate School of Law at Kobe University (020006) and the Office of Research Compliance at the University of South Carolina (Pro00105857) before it was run. The survey was conducted in Canada from 20/11/2020 to 24/11/2020, in Japan on 21/11/2020, and in the United States from 23/11/2020 to 24/11/2020. On the first page of the online survey (on Qualtrics), we provided respondents with all the necessary information about our study to ensure informed consent. This includes the researchers’ names, affiliations, and contact information; the purpose of the survey; a general explanation of what participation entails; benefits to participants (e.g., compensation); and the anonymous and voluntary nature of the survey. After reviewing this information, respondents were asked to confirm their informed consent. Additionally, there was no use of deception or misrepresentation.

Validating the conjoint design

While the general design of conjoint experiment has been validated to “work” [47], incorporating attributes from Kreps et al.’s work into our experiment helps evaluate participant engagement with our specific conjoint presentations [10]. Recovering experimental effects documented in previous work is particularly important for our study since our survey includes Canada and Japan in addition to the United States, two countries (to our knowledge) not used in conjoint designs on this subject at the time, and because we measure vaccination intentions in a distinct manner from previous conjoint-based work. Table 2 gives the results from ordered-probit models with errors clustered by respondent. In our dataset, each row represents a vaccine evaluated by a respondent, with the ordinal outcome being the speed at which they would choose to take the vaccine. The table omits the coefficient on demographic variables, which can be found in S2 Table.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Estimates for vaccine uptake models.

https://doi.org/10.1371/journal.pgph.0005410.t002

Recovering the findings in Kreps et al.’s study [10], we find that changes in the probability of minor side effects do not significantly impact decisions, whereas vaccine efficacy and the risk of severe side effects do. In terms of country of origin, the results align closely across the three survey countries, with Germany, the U.K., and the U.S. being perceived similarly in contrast to China (excluded category), but with minimal differences among themselves. Interestingly, unlike in Kreps et al’s study, we do not observe a consistently strong preference for vaccines with longer durations of effectiveness [10]. This discrepancy might be attributed to the higher COVID-19 case numbers during our survey period compared to when Kreps et al. conducted their experiment (July 2020) [10]. Overall, these findings suggest that respondents were attentive to the vaccine attributes in our design and responded in ways that are consistent with previous research. Illustrative substantive effect size simulations can be seen in S1 Fig, obtained in a manner similar to how we calculate them for endorsers described below.

Simple endorsements

We first consider the effect of a WHO endorsement on people’s willingness and speed to take the vaccine and compare it to endorsements from other actors active in global health governance. Table 2 from above presents the coefficient estimates for each of the four potential endorsers. Compared to an absence of endorsement, a WHO endorsement statistically significantly reduces the time until one takes the vaccine across all three survey countries, even after accounting for the realizations of other endorsers. In Canada, the magnitude of this effect is comparable to increasing vaccine efficacy from 50% to 90% although the effects are somewhat smaller in Japan and the United States. Similar patterns are observed for endorsements from the CDC, Oxford, and the Gates Foundation across the three survey countries.

These results are significant for two reasons. First, they demonstrate that each endorser’s support can accelerate vaccination intentions, contributing to the quicker resolution of a health crisis. This is a notable extension of previous research, which focused on general willingness to vaccinate without differentiating timing [10,12,13]. Second, the findings suggest that each endorser’s influence remains potent even in the presence of multiple (non-)endorsements, suggesting that prior endorsement results are robust to alternative designs.

To effectively compare the impact of WHO endorsements with other entities’, we need to examine the magnitudes of their effects. However, interpreting effect sizes based on coefficients in ordered probit models is not straightforward. Therefore, we use simulations of substantive effects [48]. Specifically, we calculate the differences in probabilities for each outcome choice that result from altering one attribute at a time, while averaging over all other vaccine attributes and demographic factors. Fig 1 illustrates these probability changes for both the “late” and “early” vaccination categories across each country (represented in the columns of panels), based on the three models in Table 2. We omit the middle category from this visual presentation for clarity and simplicity as it is just the remainder.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Substantive effects of endorsements on willingness to take vaccine.

This figure shows the effects when a given entity endorses a vaccine compared to when it does not do so, showing the substantive effect of endorsement relative to when an individual indicates they will get vaccinated. The figure is constructed like S1 Fig.

https://doi.org/10.1371/journal.pgph.0005410.g001

In Fig 1, the x-axis displays changes in the probability of a respondent choosing “late” (gray dot/ line) or “early” (black) when there is an endorsement from each entity on the y-axis, averaging over all covariates. The results show that the WHO endorsements reduce the “late" probability and increases the “early" probability by similar amounts. These effects are most pronounced for Canada, followed by the United States. In Canada, the effect of WHO endorsements is the greatest compared to the other three, whereas it ranks among the smallest in Japan and the United States. Overall, these results provide no consistent evidence that the WHO is the most critical or influential endorser at least in terms of encouraging vaccine uptake. Other entities, such as Oxford University, the Gates Foundation, and the CDC demonstrate similar or sometimes greater influence. All can contribute to increases in vaccination intentions and speed.

WHO endorsement and other endorsements

Next, we study whether WHO endorsements are more or less consequential under varying numbers of endorsements from other entities. To examine this, we first replace the endorsement indicators for the CDC, Oxford, and the Gates Foundation with a cumulative count of their endorsements. This sum is then included alongside an interaction term with the WHO endorsement. Table 3 presents the coefficient estimates for the WHO endorsement, the number of endorsements from other entities, their interaction, and other vaccine features; the full table of coefficient estimates is available in the S4 Table.

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. Estimates for vaccine uptake models interacting WHO endorsement and number of other endorsers.

https://doi.org/10.1371/journal.pgph.0005410.t003

Across the three survey countries, we find a diminishing effect of WHO endorsements as the number of endorsements from other entities increases, as indicated by the positive coefficient on the interaction term. This suggests that WHO endorsements are substitutable—being more influential when there are fewer endorsements from other entities. Conversely, the presence of other endorsers can diminish the WHO’s importance, a case in which the WHO endorsement is less needed (as indicated by our previous analyses), however. This pattern is more distinct in Canada and the United States, while in Japan, the estimates are less clear-cut.

Fig 2 shows the substantive effects of the WHO endorsement on each of the outcome probability (y-axis) conditional on the number of other endorsements (x-axis). In Canada and the United States, we clearly observe the effect moderation by the number of other endorsements. When no other entities endorse a vaccine, the WHO’s endorsement increases the probability of getting vaccinated “early” and decreases the probability of “late” vaccination. However, as more endorsements are added along the x-axis, the influence of the WHO on vaccination intentions diminishes. Notably, the substantial effect of the WHO endorsement actually becomes indistinguishable from zero when the CDC, Oxford University, and the Gates Foundation jointly endorse a vaccine. In Japan, people react rather little to the WHO endorsement conditional on any number of other endorsers. This suggests that the absence of an effect for Japan in Fig 1 was not due to effect heterogeneity with regard to the number of other endorsers.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. WHO effects conditional of the number of other endorsements.

This figure shows how multiple endorsers affect when people will get vaccinated. Given WHO endorsement, the figure shows change in probability for taking the vaccine early (gray) or late (black) based on the number of co-endorsers.

https://doi.org/10.1371/journal.pgph.0005410.g002

Our findings, particularly in Canada and the United States, suggest that the WHO’s endorsement can make the largest difference when other entities are slower to examine and endorse a vaccine. Conversely, if the WHO is slow to endorse, other entities can effectively step in to fill the gap. More generally, this substitutability implies that a multitude of endorsements or a consensus among various health-focused organizations is not necessarily crucial for vaccine promotion. Instead, the key is to ensure that the first or first few credible entities take prompt action, and the WHO can be key if it moves fast and early.

To illustrate this point, we simulate the expected share of vaccinated people—quantities that are more relevant from a public health perspective—as time progresses under different endorsement scenarios: with and without the WHO’s endorsement and with zero or three other endorsements. We reestimate the models from Table 3 but rely on the full range of outcome options so that we can show the vaccinated shares over time. For every observation in the data, we simulate when they would take each type of vaccine, which lets us calculate the share of vaccinated people (measured by stated vaccination intention) at each point in time. Relying on the parametric bootstrap, we obtain measure of uncertainty based on the model.

In Fig 3, the y-axis represents the share of vaccinated individuals at a particular time point (x-axis). For example, in Canada, when there are no other endorsements (upper panel), about 18% of the population will have taken the WHO-endorsed vaccine (gray) within a month, 32% within 2-3 months, 51% within 4-12 months, and 69% after a year. When the WHO does not endorse the vaccine (black), these shares drop significantly, aligning with our earlier discussion.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. WHO effects on the share of vaccinated people when there are no or two other vaccine endorsements.

In each row of panels, the x-axis gives the time points and the y-axis shows the share of vaccinated people when there is WHO endorsement (gray) and when there is not (black). The upper panels give the results assuming no other vaccine endorsements, the bottom assuming three others. The dot gives the median estimate, the line 68% confidence interval (about one standard deviation).

https://doi.org/10.1371/journal.pgph.0005410.g003

However, when there are endorsements from three other entities (lower panel), the vaccination rates reach about the same as when there is only a WHO endorsement (gray in upper panel). This occurs even without the WHO endorsement because the other two endorsements have already increased vaccination rates while reducing the impact of an additional WHO endorsement. This demonstrates the significant role of timely endorsements from credible entities in promoting vaccination and the diminishing impact of additional endorsements once initial credibility is established.

WHO endorsement for low-quality vaccines

Conjoint analyses of vaccination intentions are powerful because they allow us to study a large space of possible vaccines. Of course, the vast space of potential vaccines does not exist in reality among the vaccines that make it into medical trials. In November 2020, the world learned the remarkable effectiveness of mRNA vaccines against COVID-19 with minuscule side effects. When vaccine trials started earlier that year, the U.S. Federal and Drug Administration said it would be willing to approve a vaccine with 50% effectiveness against severe illness, a benchmark that was eclipsed by the Phase 3 trial results by BioNTech/Pfizer and Moderna vaccines in November 2020 [49].

Vaccines developed for future pandemics may not always be highly effective, particularly those rapidly produced in the early stages of an outbreak due to urgent public health demands. Existing research examining vaccination intentions using vaccine attribute profiles consistently finds that respondents prefer higher-quality vaccines, reducing the relative importance of endorsements in such cases. Consequently, it is crucial to examine the influence of WHO endorsements specifically for lower-quality vaccines, as vaccination intentions for these vaccines are likely to be more sensitive to credible endorsements.

We use simulations based on the models without interactions and with the full range of outcomes to illustrate the influence of WHO endorsements on the share of vaccinated people over time, as shown in Fig 4. We define low-quality vaccines as those with shorter protection duration (1 year), lowest efficacy (50%), and higher odds of side effects (1 in 10,000 and 1 in 30 for severe and mild side effects, respectively). We average over the randomized country of origin and endorsements by the CDC, Oxford, and Gates foundation.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. WHO effects on the share of vaccinated people for low-quality vaccine.

The x-axis gives the time points and the y-axis shows the share of vaccinated people when there is WHO endorsement (gray) and when there is not (black). The dot gives the median estimate, the line 68% confidence interval (about one standard deviation).

https://doi.org/10.1371/journal.pgph.0005410.g004

Fig 4 presents simulation results showing vaccination shares over time, comparing scenarios with a WHO endorsement (gray) to those without (black) like before. In Japan, the WHO endorsement has little effect on vaccination rates at each time point for low-quality vaccines, consistent with previous observations. In the United States, there is a small effect on vaccination rates that gradually increases over time, though the increase is noisy. The Canadian case is particularly instructive. When people are receptive to WHO endorsements, we observe a distinctly higher vaccination rate over time. After a year, there is a ten percentage-point increase in the final share of individuals opting for the WHO-endorsed vaccine.