https://orcid.org/0000-0002-7559-1592
Figures
Abstract
The COVID-19 pandemic has been an historic challenge to public health and behavior change programs. In low -and middle-income countries (LMICs) such as Nigeria, there have been challenges in promoting vaccination. Vaccine hesitancy and social norms related to vaccination may be important factors in promoting or inhibiting not only COVID vaccination, but other routine vaccinations as well. The aim of this study was to conduct a national-level quasi-experimental evaluation of a social media based COVID-19 vaccination promotion campaign in Nigeria run in 2022. We followed a longitudinal cohort of Nigerians (at baseline) drawn from all 37 states in Nigeria over a 10-month period. This was done at 3 time points to evaluate psychosocial predictors of vaccination and vaccination outcomes following a theory of change based on Diffusion of Innovations, Social Norms Theory, and the Motivation, Opportunity, Ability (MOA) Framework. In a quasi-experimental design, participants in 6 Nigerian states where the social media campaign was run (treatment) were compared to participants from non-treatment states. This study highlights new social media-based data collection techniques. The study found that vaccination rates increased in treatment states compared to non-treatment states, and that these effects were strongest between baseline and first follow up (December 2021 to March 2022). We also found that more pro-vaccination social norms at one time point are associated with higher vaccination rates at a later time point. Social media campaigns are a promising approach to increasing vaccination at scale in LMICs, and social norms are an important factor in promoting vaccination, which is consistent with the Social Norms Theory. We describe implications for future vaccination campaigns and identify future research priorities in this area.
Citation: Evans WD, Bingenheimer JB, Long M, Ndiaye K, Donati D, Rao NM, et al. (2023) Outcomes of a social media campaign to promote COVID-19 vaccination in Nigeria. PLoS ONE 18(9): e0290757. https://doi.org/10.1371/journal.pone.0290757
Editor: Naeem Mubarak, Lahore Medical and Dental College, PAKISTAN
Received: June 1, 2023; Accepted: August 2, 2023; Published: September 15, 2023
Copyright: © 2023 Evans et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: *****PA AT ACCEPT: Please follow up with authors for repository information available at Accept***** PLEASE NOTE THAT WE STATED WE WILL PROVIDE ACCESS TO OUR DATA UPON REQUEST, NOT THAT WE WILL PROVIDE REPOSITORY INFORMATION. THUS WE WILL NOT BE PROVIDING ACCESSION NUMBERS OR DOIs AT THIS TIME.
Funding: The study was funded through INV-033413 from the Gates Foundation. We also state that the project officer who was an employee of the funder at the time this project was initiated, Dr. Sohail Agha, participated in discussions about the study design and data collection. At the time of this manuscript preparation, Dr. Agha was no longer with the funder. Dr. Agha did participate in development of the manuscript as stated in the author roles section.
Competing interests: The authors have declared that no competing interests exist.
Introduction
The COVID-19 pandemic has led to over seven million deaths worldwide as of June 2023, and has been a monumental challenge for medical and public health systems worldwide. The introduction of COVID-19 vaccines in December 2020, and rollout of vaccination programs in the following years have been critical in controlling the pandemic, but have also led to the challenge of vaccine hesitancy [1, 2]. The existence of mis- and disinformation, especially online, has created a major barrier to vaccine acceptance and uptake [3]. Thus, there is need to create demand and to understand the factors that promote and inhibit vaccine uptake in groups that are either uninformed, vaccine hesitant, or actively resistant to COVID-19 vaccination.
COVID-19 is an historic global health challenge, and both supply and demand for vaccination in Low- and Middle-Income Countries (LMICs) is a major issue in controlling the pandemic worldwide. Specific Issues in LMICs include the 5 Cs of vaccine hesitancy (confidence, complacency, convenience, communications, and context), social norms around vaccination, mis- and disinformation, among other challenges [4]. Additionally, variables such as gender, education levels, socio-economic status, and endorsement by trusted actors such as Health Care Providers (HCPs) have been found to affect vaccine hesitancy and uptake in LMICs [5].
Positive role models who promote a pro-vaccination norm, such as HCPs, business leaders, community and religious leaders, government officials, celebrities, and others are an important potential source of social influence to promote vaccination [6]. Demonstrating the benefits of vaccination and overcoming false and misleading information (e.g., about negative health effects of vaccination, such as infertility) through influential messaging is a potentially powerful avenue to increase vaccine uptake.
Vaccine hesitancy among important potential role models such as HCPs may be a major barrier to the uptake of COVID-19 vaccines in large LMICs such as Nigeria. This is a focus on vaccine uptake efforts especially given the observed role of mis- and disinformation about COVID-19 vaccination in many countries during the pandemic [7]. HCPs have been identified by the Nigerian government as a key target population for COVID-19 vaccinations. As HCPs will be responsible for the rollout of vaccines in the general population of Nigeria, their lack of adoption of COVID-19 vaccines may hamper the rollout of the national campaign. Studies in high-income countries (HICs) show that vaccinated HCPs are more likely to be successful in persuading communities to receive vaccines. There is also some concern that, if HCPs exhibit vaccine hesitancy, vaccines allocated to them may get diverted to other populations. By modeling vaccine acceptance behavior, HCPs are well-positioned to influence norms of behavior amongst their colleagues. There is some evidence from HICs on effective interventions to reduce vaccine hesitancy, but much more research is needed on effective strategies and messages [8–10].
The present study aims to evaluate the effects of a social media campaign to promote COVID-19 vaccination in Nigeria. The campaign was based on a Theory of Change (ToC) that posited that promoting positive social norms in favor of vaccination, reducing vaccine hesitancy, and promoting pro-vaccination Motivation, Opportunity, and Ability (MOA) to be vaccinated would collectively lead to higher vaccination rates [11]. The ToC for this campaign was based on Diffusion of Innovations, Social Norms Theory, MOA framework [12–14]. The ToC specified primary and secondary outcomes in terms of COVID-19 vaccination, vaccine hesitancy, and the psychological and population-based processes by which changes in these outcomes occur. This ToC was designed as a basis for a social media campaign in Nigeria using the Facebook and Instagram platforms, which have large followings in the country. The ToC hypothesized a set of mediating variables, which included social norms, role modeling, and cultural beliefs that are theoretically related to vaccine intentions and vaccination behavior [15].
This study reports on findings from a national evaluation of the campaign in all 37 states in Nigeria using a quasi-experimental design. We tested two primary hypotheses (H). First, H1: Exposure to the social media campaign in treatment states will produce higher rates of COVID-19 vaccination compared to non-treatment states. Second, H2: Lower levels of vaccine hesitancy at earlier time points will be associated with higher vaccination rates at later time points, and similarly higher levels of pro-vaccination social norms at earlier time points will be associated with higher vaccination rates at later time points.
Methods
Design and intervention
The current study was part of a larger evaluation of the impact of social media campaigns to promote COVID-19 vaccination among HCPs and others in their social environment in Nigeria. The evaluation employed mixed methods and comprised of a quantitative study conducted through social media-based surveys, a qualitative study of stakeholders, and a cost effectiveness study. The qualitative and cost effectiveness study findings are reported elsewhere. Within the quantitative study, we collected baseline data from participants across Nigeria recruited through a Facebook-based survey, protocol described below, in December 2021, March-April 2022, and October 2022.
The campaign was designed and delivered by a separate team of designers and local organizations in Nigeria who created social media content and delivered it on Facebook and Instagram. Local organizations included healthcare organizations, businesses, religious groups, and similar prominent leaders with a substantial, existing social media presence and network. Content consisted of graphics, videos, and other social media materials such as GIF (graphic interface format) images. The campaign was run in two phases, from January to February 2022, and from May to September 2022, corresponding to the time periods just after baseline, before the first follow up, the period after the first follow up, and just before the second (endline) follow up.
The evaluation was reviewed and approved by The George Washington University Institutional Research Board (IRB) and by the National Health Research Ethics Committee (NHREC) in Nigeria. Consent was obtained by providing a consent statement to participants in the social media chatbot (described below). Participants who clicked to continue the survey after reading the statement consented to participate.
Vaccine hesitancy occurs along a continuum between full acceptance, including high demand for vaccines, and outright refusal of some or all vaccines. It is widely accepted that communications campaigns are best used to target people in the center of a spectrum—the ‘persuadable middle’. The campaign therefore attempted to reach HCPs and their communities who were unsure or delaying getting vaccinated.
Using the previously described ToC as a basis, the campaign designers iterated a series of social media content focused on promoting pro-vaccination social norms and reducing vaccine hesitancy using social influencers (individuals such as local celebrities, other HCPs, religious and business leaders, and other individuals who would be respected and trusted by the audience). The campaign ToC was premised on the idea that a social media campaign that achieved high levels of reach and audience engagement would be able to reduce vaccine hesitancy, increase pro-vaccination social norms, and related attitudes and beliefs, thus increase COVID-19 vaccination uptake.
Measures
The focal independent variable in this study was treatment arm, meaning whether the respondent’s Facebook account was assigned to a campaign state or a comparison state. We constructed a treatment state indicator variable on which respondents from campaign states (Anambra, Bauchi, Lagos, Niger, Rivers, or Sokoto) received a code of 1, and those from any of the remaining states or the Federal Capital Territory received a code of 0. The campaign states were selected by the campaign design team.
The primary endpoint in this study is COVID-19 vaccination uptake. This was assessed using a single questionnaire item which read, “Have you received a COVID-19 vaccine?” There were four response options: “Yes, a single-dose vaccine”; “Yes, the first dose of a two-dose regimen”; “Yes, both doses of a two-dose regimen”; and “No”. Due to changes over the course of the study in the availability of different types of COVID-19 vaccines in Nigeria as well as recommendations for receiving boosters, we combined the first three responses into a single vaccinated category (coded 1), while those who answered “No” were coded 0 for unvaccinated.
This study also had two secondary endpoints. The first was vaccine hesitancy, an index defined as the mean of coded responses to five questionnaire items reflecting the Five Cs framework and adapted from [16]. The items read, “I am confident that COVID-19 vaccines are safe and effective,” “Vaccination against COVID-19 is unnecessary,” “Everyday stress prevents me from getting a COVID-19 vaccine,” “When I think about getting vaccinated against COVID-19, I weigh the benefits and risks to make the best decision possible,” and “When everyone is vaccinated against COVID-19, I don’t have to get vaccinated too.” These were intended as measures of confidence, complacency, convenience, calculation, and collective responsibility, respectively. For each of these items, respondents were asked to select one of the following five response options: “Strongly agree,” “Agree,” “Neither agree nor disagree,” “Disagree,” and “Strongly disagree.” We coded these from 1 to 5 or 5 to 1 depending on the wording of the item (i.e., whether agreement reflected lower or higher levels of hesitancy). The resulting index had low internal consistency (Cronbach’s a = 0.33 at baseline), but this is not surprising because the items represent diverse sources of hesitancy rather than serving as indicators of a single, unidimensional underlying construct. Because it is defined as the mean of component items, our vaccine hesitancy index has a plausible range of 1 to 5.
The study’s other secondary endpoint was pro-vaccination social norms, a scale score defined as the mean of coded responses to five questionnaire items adapted from previously validated measures [17]. The first two items read, “Your friends think it is important for everyone to get a COVID-19 vaccine” and “Your family members think it is important for everyone to get a COVID-19 vaccine.” The response options for each were “Strongly disagree,” “Disagree,” “Neither agree nor disagree,” “Agree,” and “Strongly agree,” and we coded these 1 to 5. The remaining three questionnaire items assessing social norms read, “Of the people close to you, what proportion of them would want you to get the COVID-19 vaccine?”, “How many people in Nigeria do you think will get the COVID-19 vaccine when it becomes available?”, and “How many people who work in healthcare in Nigeria do you think will get the COVID-19 vaccine when it becomes available?” The response options for these were, “A few (1–20%),” "Some (21–40%),” “Many (41–60%),” “Most (61–80%),” and “Nearly all or all (81–100%).” We coded these 1 to 5, respectively. The content of these items reflects two key elements in the theory of social norms and the application thereof to health promotion interventions [16]: the distinction between descriptive norms (perceptions of what others do) and injunctive norms (perceptions of what others think one should do) [18, 19], and the potential relevance of multiple reference groups for any given health-related behavior [17, 20]. The first two items capture injunctive norms, while the third, fourth, and fifth capture descriptive norms. And together, the five items capture five different reference groups. The resulting scale, defined as the mean of the component items, had a plausible range of 1 to 5 and a moderately high internal consistence (Cronbach’s a = 0.74 at baseline).
Data collection
Survey data for the current study were collected on a social media-based research platform called Virtual Lab (https://vlab.digital/) [21]. The survey instrument was designed to measure the previously mentioned outcomes, as well as key elements in the campaign ToC, described earlier.
Virtual Lab is an open-source software platform that uses targeted digital advertising to recruit a custom sample of respondents. In this case, the study was stratified by health care worker status, and we sought to obtain 50% of our total sample in this group with the rest being the general Nigerian population. Sample size estimates aimed to detect a change in COVID-19 vaccination rates among HCPs and the general population. To be eligible, targeted participants had to be 18 years of age or older, have a Facebook account, and they received recruitment advertising in their live feed promoting a study on COVID-19 vaccination. Facebook is the 2nd leading social media platform in Nigeria (after WhatsApp), and mobile phone use in the country is the highest in Africa, enabling nationwide data collection [22]. By clicking through the ad, potential participants then engaged with a chatbot that used Facebook Messenger (FM) to send reply chats. A series of FM chats then screened and offered an opportunity for participants to provide informed consent statement following the IRB-approved protocol. Screening questions included age and health care worker status and job role, if applicable. After reading the consent statement, agreement to proceed with the survey represented consent. Eligible participants were offered an incentive of 400 Naira (about $1USD) in mobile phone credits to complete a questionnaire consisting of 40 items, each delivered as an individual FM message. Data were captured and stored in a secure Virtual Lab database.
Individuals who consented to participate were sent questionnaire items one at a time via the chatbot application. For most items, participants were prompted simply to tap on the response option of their choice to answer that item. After answering each item, participants received the next item within a few seconds, and this process was repeated through the end of the questionnaire. Responses were collected in an electronic database and exported as a.csv file for analysis.
Data analysis
For our primary endpoint of COVID-19 vaccine uptake, we first obtained frequencies and percentages at each wave by study arm. Then, for vaccination status at first and second follow-up, we ran two linear regression models, one with the treatment state indicator as the sole independent variable, and the other with the following control variables: age group, gender, education, religion, and occupation, all assessed at baseline. We used clustered standard errors for statistical inference in these models to account for the nesting of participants within states.
Similarly, for our two secondary endpoints of vaccine hesitancy and pro-vaccination social norms, we obtained descriptive statistics at each wave by study arm, and then used linear regression to test for differences between treatment and comparison state residents. For these outcomes, we included baseline levels of the outcome variable as an additional control variable in the adjusted models. As with our analysis of vaccination status, we used clustered standard errors to account for the nesting of participants within states.
We next conducted logistic regression analysis to test two hypotheses from the campaign’s theory of change: that vaccine hesitancy decreases vaccine uptake, and that pro-vaccination social norms increase vaccination uptake. We did this first using baseline levels of vaccine hesitancy and social norms to predict vaccination uptake between baseline and first follow-up; and then using vaccine hesitancy and social norms from first follow-up to predict vaccination uptake between first and second follow-ups among those who were not yet vaccinated at first follow-up. For each pair of time periods, we ran four logistic regression models: one with vaccine hesitancy as the sole independent variable, the second with social norms as the sole independent variable, the third with both vaccine hesitancy and social norms as independent variables, and the fourth with controls for age group, gender, education, religion, and occupation. As before, we used clustered standard errors to correct standard errors and p-values for the nesting of participants within states.
Next, given the high level of attrition over the course of the study, we conducted an analysis of patterns of attrition. We created a 0-or-1 indicator variable for attrition before first follow-up and attrition before second follow-up. We cross-tabulated each of these with our treatment state indicator variable, age group, gender, education, religion, and occupation. We also created categorical versions of our baseline vaccine hesitancy index and social norms variables, with scores between 1 and 2 categorized as “lowest,” scores between 2 and 3 categorized as “low,” scores between 3 and 4 categorized as “high,” and scores between 4 and 5 categorized as “highest.” We cross-tabulated these with our attrition indicators, just as we did for the treatment arm and the socio-demographic variables. For each potential determinant of attrition, we also estimated a logistic regression model (also using clustered standard errors) to determine whether that variable’s association with attrition was statistically significant. We also tested for differential attrition across arms using logistic regression models with the treatment state indicator variable, one of the baseline variables (age, gender), and the treatment indicator by baseline variable interaction terms. We used likelihood ratio tests of nested models (with versus without the interaction terms) to test whether the pattern of attrition was differential for each baseline variable.
Finally, as robustness checks, we conducted two alternative versions of our main analyses. In the first, we used carry-forward imputation for missing values of vaccination status, vaccine hesitancy, and pro-vaccination social norms at first and second follow-ups. Because being vaccinated at enrollment was an exclusion criterion, this meant that everyone who was lost to follow-up was assumed to still be unvaccinated at those follow-up time points in these analyses. We repeated the primary analyses (described in the first two paragraphs of this section) using the dataset with carry-forward imputation for missing values. Next, we used multiple imputation by chained equations [23, 24] to create 50 completed datasets. Input variables for the imputations included all baseline demographic variables as well as the baseline vaccine hesitancy index and pro-vaccination social norms scale. We then carried out analyses analogous to our primary ones using these imputed datasets and obtained aggregated point estimates and standard errors following the formula derived by Little and Rubin [25]. All data management and analysis were carried out in Stata version 17 [26]. Clustered standard errors were obtained by adding the option vce(cluster state) to all linear and logistic regression commands. For the multiple imputations and the analyses of the imputed datasets we used Stata’s mi impute chained command and regression commands with the mi estimate prefix.
Results
Fig 1 is the CONSORT diagram that shows the flow of (potential) participants into and through the study. A total of 10,965 potential participants received links to the Facebook Messenger questionnaire and began answering screening questions. Over 82% did not meet the eligibility criteria, with already having received a vaccination against COVID-19 accounting for the majority of exclusions, followed by not being in the “persuadable middle” in terms of intention to get vaccinated. The persuadable middle was defined as those who responded that they were not already vaccinated were asked, “Would you take a COVID-19 vaccine that is approved for use in Nigeria if offered to you?” Those who responded “Definitely” or “Definitely not” were not in the persuadable middle. Additional exclusions included missing baseline variables, being below age 18, and having a duplicate ID number (which occurred when someone from a single Facebook account linked to the questionnaire more than once). We thus enrolled 1,100 participants from the six campaign (treatment) states and 833 from the 31 comparison states. Of these, 1,155 (59.8%) completed the first follow-up questionnaire, and 462 (23.9%) completed the second follow-up questionnaire.
Table 1 shows the composition of the sample at baseline. The sample was young, with over 60% being under age 30. The majority were men, and more than half had earned a secondary school diploma or higher. The most common religious affiliation was non-Catholic Christian, followed by Muslim, Roman Catholic, and others. Just over two-thirds were not HCPs, and those who were HCPs were distributed across a variety of specific occupations. Some differences in the socio-demographic composition from participants from treatment versus comparison states are evident. S1 Table in S1 File shows the distribution of treatment state participants across the six treatment states, and of comparison state participants across the 31 comparison states. Notably, one state, namely Lagos, contributed to the vast majority of treatment state participants, whereas comparison state participants were somewhat more widely distributed across the comparison states.
The main findings for primary and secondary endpoints at the first and second follow-up questionnaires are presented in Table 2. At first follow-up, 31.7% of retained participants from treatment states, compared to 25.3% of retained participants from comparison states, for a crude difference of 6.4 percentage points (p = 0.045) and an adjusted difference of 7.8 percentage points (p = 0.020). At second follow-up, these figures increased to 44.1% and 35.0% for retained treatment and comparison state participants, respectively. The crude and adjusted differences at second follow-up in percentage point terms were 9.1 and 11.0, but these were not statistically distinguishable from the null (p = 0.076 for the crude and p = 0.068 for the adjusted difference), although these changes approach the level of statistical significance. Vaccine hesitancy scale scores decreased slightly on average among treatment state participants, but also decreased among participants from comparison states, and neither the crude nor the adjusted differences at either first or second follow-up were statistically distinguishable from the null. Scores on the pro-vaccination social norms scale increased very slightly between waves in participants from treatment and control states, but none of the crude or adjusted differences were statistically significant. S2 Table in S1 File provides a more detailed look at vaccine hesitancy and pro-vaccination social norms at the item rather than scale level, where crude and adjusted differences at both follow-up time points were almost universally statistically indistinguishable from the null.
Table 3 presents results of tests of hypotheses that vaccine hesitancy at one point in time would be negatively associated with uptake of COVID-19 vaccination before the next time point, and that pro-vaccination social norms would be positively associated with vaccination uptake over the same periods. Across all of these models, none of the crude or adjusted odds ratio for the relationship between vaccine hesitancy and vaccination uptake are statistically significantly different from the null, although most are in the hypothesized negative direction. In contrast, all of the crude and adjusted odds ratios for pro-vaccination social norms are statistically significant, with point estimates that suggest that a unit increase in pro-vaccination social norms increases the odds of being vaccinated between waves by 50 to 60 percent.
Table 4 presents results of a detailed analysis of patterns of attrition between baseline and first follow-up, and between baseline and second follow-up. Overall levels of loss to follow-up were similar among participants in treatment and comparison states. But several sociodemographic variables were statistically significantly associated with being lost to follow-up rather than retained. For example, participants in the oldest age category were substantially more likely to be lost to follow-up than those in younger age categories, and medical doctors and pharmacists were more likely than participants in other health sector and non-health fields to be lost to follow-up. Loss to follow-up also differed in relation to baseline vaccine hesitancy, with participants at the lowest and highest levels of the vaccine hesitancy scale being more likely than those in the middle ranges of vaccine hesitancy. Moreover, several of the study arm by background variable interactions were statistically significant, suggesting that the pattern of attrition may have been different among participants from treatment states than among those from comparison states.
Findings of sensitivity analysis intended to address the large magnitude and differential pattern of loss to follow-up are presented in Table 5. Panel A shows findings from carry-forward imputation analyses. For vaccination uptake, the point estimates are in the same direction as those presented in Table 2, with vaccination uptake being more common among participants from treatment states than from comparison states. While these differences are not statistically significant at first follow-up, they are statistically significant at second follow-up. For vaccine hesitancy index scores, crude and adjusted differences at first and second follow-ups remain small, but one–the crude difference at second follow-up–is now statistically significant and favors treatment state participants. Crude and adjusted differences in pro-vaccination social norms scale scores are likewise small, and the one difference that is statistically significant–the crude difference at second follow-up–actually favors comparison state participants. Panel B shows findings from multiple model-based imputation. None of the differences between treatment and comparison state participants are statistically significantly different from the null in these analyses, but the point estimates for vaccination uptake are in the positive direction and are close to being statistically significant in the models for vaccination uptake a first follow-up.
A. First Robustness Check Results: Levels of Vaccination Uptake, Vaccine Hesitancy, and Pro-Vaccination Norms, and Crude and Adjusted Differences at First and Second Follow-Ups with Simple Carry-Forward Imputation. B. Second Robustness Check Results: Levels of Vaccination Uptake, Vaccine Hesitancy, and Pro-Vaccination Norms, and Crude and Adjusted Differences at First and Second Follow-Ups with Multiple Model-Based Imputation via Chained Equations.
Discussion
This study aimed to evaluate the effects of a large-scale social media campaign to promote COVID-19 vaccination in Nigeria using a quasi-experimental design. Given that there have been relatively few of such large-scale evaluations of social media to promote behavior change, especially in LMICs, this research has the potential to improve the evidence base in the field [27]. The study tested a set of hypotheses based on a grounded ToC, which has been absent from many previous social media campaigns in LMICs [27].
The main overall finding confirms H1: There is evidence that the campaign promoted higher levels of vaccination in treatment states compared to comparison states. As stated earlier, both the crude and adjusted models reflect a positive effect of the campaign on retained participants in treatment states compared to control states. At the first follow-up, 31.7% of retained participants from treatment states had been vaccinated, compared to 25.3% of retained participants from comparison states who had been vaccinated, for a crude difference of 6.4 percentage points (p = 0.045) and an adjusted difference of 7.8 percentage points (p = 0.020). While the magnitude of these differences were even greater at the second follow up, there were 9.1 and 11.0 percentage points of higher vaccination among retained treatment state participants compared to comparison state participants; the results approached but did not reach the level of statistical significance, defined as P < .05.
It is worth noting that we had substantial loss to follow up (LTFU) by the time of the second follow up. While we did not observe differential attrition by treatment state versus comparison state, some differences in LTFU were observed between demographic characteristics, and by vaccine hesitancy status (i.e., the extreme responses were most likely to be LTFU, as compared to the middle of the hesitancy range). The substantial LTFU, while not different by study condition, may have reduced power to detect an effect of treatment on vaccination rates at the second follow up, and this should be considered in interpreting the near significant effects in the treatment states at the second time point.
The secondary analyses partially confirm H2: While we did not find that vaccine hesitancy at one time is associated with higher vaccination at a later time point, we did find evidence that more pro-vaccination social norms at one time point are associated with higher vaccination rates at a later time point. As stated earlier, none of the crude or adjusted models show a statistically significantly relationship between vaccine hesitancy and vaccination uptake, although most are in the hypothesized negative direction (i.e., higher vaccine hesitancy is associated with lower vaccination rates). However, all of the crude and adjusted odds ratios for pro-vaccination social norms are statistically significant, and the models suggest that a unit increase in pro-vaccination social norms increases the odds of being vaccinated between waves by 50 to 60 percent. This suggests that pro-vaccination normative beliefs may be an important driver of COVID-19 vaccination.
Similar overall considerations in terms of LTFU should be considered for the secondary outcomes. We observed some differences in attrition for very low and very high vaccine hesitancy respondents, and this may have affected the results for the relationship between vaccine hesitancy at an earlier time point and vaccination status at a later time point. We did not observe similar attrition differences based on social norms scale responses.
Overall, we conclude that there is evidence that the campaign produced measurable and practically meaningful positive effects on social norms and vaccination status. The fact that social norms were hypothesized as one mediator of the effects on vaccination provides support for the campaign’s ToC. Further analysis of the potential mediating effects of social norms on vaccination should be conducted, and future research should examine the potential for social norms to be a mediator of COVID-19 and other vaccine uptake.
One future avenue for new interventions is to identify normative beliefs that increase vaccine hesitancy or otherwise serve as barriers to getting vaccinated, such as motivations and beliefs about opportunities and ability to get vaccinated (per the MOA framework) [28, 29]. Myths and misinformation about the effects of COVID-19 and other vaccines (e.g., that they cause infertility) are likely part of the mosaic of normative beliefs that increase vaccine hesitancy, and should be addressed through messaging and other intervention strategies [3, 30].
Recommendations for future research include employing stronger quasi-experimental designs such as cluster randomization in future studies on social media programs to promote vaccination. While circumstances on the ground in Nigeria at the time of this project prevented such an approach, randomly assigning sufficient states to treatment and control would have reduced important limitations of this study and enhanced generalizability. Further development and evaluation of outcomes measures used here, including vaccine hesitancy and social norms, should be done in the future. Finally, evaluation of the campaign theory of change should be conducted through formal mediation analysis (Structural Equation Modeling).
One area to explore further in future research on such campaigns is the mechanisms by which social norms are changed and influence vaccination outcomes via social media. One hypothesis is that social media may create a widespread sense of engagement in behaviors and social acceptance of behaviors such as COVID-19 and other forms of vaccination [31]. The mechanisms by which social media influence vaccination social norms deserve further experimental research.
Limitations of the current research
The design was non-randomized, and the treatment states were selected purposefully by the campaign design team, which limits the generalizability of findings. To address this, and to be reported elsewhere, we conducted a separate randomized controlled study of individuals in non-treatment states. We also experienced relatively high attrition. The social media-based sample was collected by convenience on the Facebook platform and is thus not nationally representative of Nigerians. Also, we did not have detailed information on possible contamination of participant awareness of campaign messages from other sources, such as Nigerian government vaccination campaigns running at the same time as our campaign and evaluation study. However, we also note that the intervention reported here was the only statewide social media campaign running in the treatment states during the evaluation. Finally, while we did not observe differential attrition by treatment condition, however, we did observe some differences in attrition rates by demographic and psychographic (vaccine hesitancy) status. As a result of these differences in LTFU, findings of this study should be interpreted with caution.
Finally, the evaluation will contribute both to the knowledge of digital interventions in LMICs, specifically in Nigeria, and also to interventions for HCPs in LMICs. The health care sector is growing rapidly along with the population in LMICs, especially in sub-Saharan Africa. Best practices to improve care include a focus on HCP behavior, and their role in promoting population health through role modeling of healthy behaviors such as vaccination. This project provides one of the first and largest examples of best practices in promoting healthy behaviors among HCPs in an LMIC setting. Findings from this project contribute to the evidence base and support the global agenda to promote vaccination in LMICs [32].
Conclusions
We found that the campaign had positive effects on vaccination status and that pro-vaccination social norms are associated with higher rates of vaccination. Social media has the potential to influence vaccination rates in LMIC settings, and there is evidence to suggest that social norms are a promising mechanism to communicate pro-vaccination messages. Further research on pro-vaccination social norms messages is needed. Additionally, there is a need to understand the quantity and frequency of social media messages that are needed to increase vaccine willingness and vaccination rates [33]. More research on dose-response effects of pro-vaccination campaigns is needed.
Supporting information
S1 File. Further details on the geographic distribution of the sample and item-level averages for the 5 Cs and social norms items.
https://doi.org/10.1371/journal.pone.0290757.s001
(DOCX)
Acknowledgments
The social media campaign evaluated in this study was led by M&C Saatchi, Upswell, and a consortium of Nigerian organizations that published the social media content with approval from the Nigerian government. We thank these organizations for their collaboration.
References
- 1. MacDonald NE. Vaccine hesitancy: Definition, scope and determinants. Vaccine. 2015 Aug;33(34):4161–4. pmid:25896383
- 2. Larson HJ, Jarrett C, Schulz WS, Chaudhuri M, Zhou Y, Dube E, et al. Measuring vaccine hesitancy: The development of a survey tool. Vaccine. 2015 Aug;33(34):4165–75. pmid:25896384
- 3. Lee SK, Sun J, Jang S, Connelly S. Misinformation of COVID-19 vaccines and vaccine hesitancy. Sci Rep. 2022 Aug 11;12(1):13681. pmid:35953500
- 4. Pheerapanyawaranun C, Wang Y, Kittibovorndit N, Pimsarn N, Sirison K, Teerawattananon Y, et al. COVID-19 Vaccine Hesitancy Among Health Care Workers in Thailand: The Comparative Results of Two Cross-Sectional Online Surveys Before and After Vaccine Availability. Front Public Health. 2022 Aug 1;10:834545. pmid:35979452
- 5. Moola S, Gudi N, Nambiar D, Dumka N, Ahmed T, Sonawane IR, et al. A rapid review of evidence on the determinants of and strategies for COVID-19 vaccine acceptance in low- and middle-income countries. J Glob Health. 2021 Nov 20;11:05027. pmid:34912550
- 6. Bhopal S, Nielsen M. Vaccine hesitancy in low- and middle-income countries: potential implications for the COVID-19 response. Arch Dis Child. 2021 Feb;106(2):113–4. pmid:32912868
- 7. Mubarak N, Khan AM, Tariq S, Kanwal S, Tariq S. The Perils of Vaccine Hesitancy Based on Conspiracy Theories and Misinformation: Implications and way Forward for Pakistan. J Univ Med Dent Coll. 2021 12(2), vii–xiv. https://doi.org/10.37723/jumdc.v12i.
- 8. Dubé E., Gagnon D., & MacDonald N. E. (2015). Strategies intended to address vaccine hesitancy: Review of published reviews. Vaccine, 33(34), 4191–4203. pmid:25896385
- 9. Berry S. D., Johnson K. S., Myles L., Herndon L., Montoya A., Fashaw S., et al. (2021). Lessons learned from frontline skilled nursing facility staff regarding COVID‐19 vaccine hesitancy. Journal of the American Geriatrics Society, 69(5), 1140–1146. pmid:33764497
- 10. Larson H. J., Jarrett C., Eckersberger E., Smith D. M., & Paterson P. (2014). Understanding vaccine hesitancy around vaccines and vaccination from a global perspective: a systematic review of published literature, 2007–2012. Vaccine, 32(19), 2150–2159. pmid:24598724
- 11. Evans WD, Gerard R, Symington L, Shaikh H, Agha S. Implementation practice models for development in low- and middle-income countries: systematic review of peer-reviewed literature. BMC Public Health. 2022 Dec;22(1):1157. pmid:35681165
- 12.
Rogers, EM. (2003). Diffusion of Innovations, 5th edition, Free Press, New York, NY.
- 13. Cislaghi B, Heise L. Using social norms theory for health promotion in low-income countries. Health Promotion International. 2019 Jun 1;34(3):616–23. pmid:29579194
- 14. Agha S. Use of a Practitioner-Friendly Behavior Model to Identify Factors Associated with COVID-19 Vaccination and Other Behaviors. Vaccines. 2022 Aug 5;10(8):1261. pmid:36016149
- 15. Joshi A, Kaur M, Kaur R, Grover A, Nash D, El-Mohandes A. Predictors of COVID-19 Vaccine Acceptance, Intention, and Hesitancy: A Scoping Review. Front Public Health. 2021 Aug 13;9:698111. pmid:34485229
- 16. Betsch C, Schmid P, Heinemeier D, Korn L, Holtmann C, Böhm R. Beyond confidence: Development of a measure assessing the 5C psychological antecedents of vaccination. PLoS One. 2018 Dec 7;13(12):e0208601. pmid:30532274
- 17. Costenbader E, Cislaghi B, Clark CJ, Hinson L, Lenzi R, McCarraher DR, et al. Social Norms Measurement: Catching up With Programs and Moving the Field Forward. Journal of Adolescent Health. 2019 Apr;64(4):S4–6. pmid:30914167
- 18. Cialdini RB, Kallgren CA, Reno RR. A Focus Theory of Normative Conduct: A Theoretical Refinement and Reevaluation of the Role of Norms in Human Behavior. Advances in Experimental Social Psychology. 1991; (24): 201–234.
- 19. Lapinsky MK, Rimal RN. An Explication of Social Norms. Commun Theory. 2005 May; 15(2): 127–147.
- 20. Hogg MA, Reid SA. Social Identity, Self-Categorization, and the Communication of Group Norms. Communication Theory. 2006 Feb;16(1): 7–30.
- 21.
Rao N, Donati D, Orozco V. Conducting Surveys and Interventions Entirely Online: A Virtual Lab Practitioner’s Manual. 2020. World Bank: Washington, DC, USA.
- 22.
We Are Social, & Hootsuite, & DataReportal. (February 13, 2023). Most used social media platforms in Nigeria as of the 3rd quarter of 2022 [Graph]. In Statista. Retrieved June 20, 2023, from https://www.statista.com/statistics/1176101/leading-social-media-platforms-nigeria/.
- 23.
Enders CK. Applied missing data analysis. New York: Guildford Press;2010. 377 p. (Methodology in the social sciences).
- 24. White IR, Royston P, Wood AM. Multiple imputation using chained equations: Issues and guidance for practice. Statist Med. 20 2011 Feb 20;30(4):377–99. pmid:21225900
- 25.
Little RJA, Rubin DB. (2019). Statistical Analysis with Missing Data, 3rd Edition, Hoboken, NJ: Wiley.
- 26.
StataCorp. 2021. Stata Statistical Software: Release 17. College Station, TX: StataCorp LLC.
- 27. Seiler J, Libby TE, Jackson E, Lingappa JR, Evans WD. Social Media-Based Interventions for Health Behavior Change in Low- and Middle-Income Countries: Systematic Review. J Med Internet Res. 2022 Apr 14;24(4):e31889. pmid:35436220
- 28. Thaker J, Richardson LM, Holmes DC. Audience segmentation analysis of public intentions to get a COVID-19 vaccine in Australia. Public Underst. Sci. 2022, 9636625221138494, epub ahead of print. https://doi.org/10.1177/09636625221138494 pmid:36546333
- 29. Saul H, Gursul D. Stressing the personal benefits of the covid-19 vaccine might encourage more people to accept it. BMJ. 2021, 375, n2923. https://doi.org/10.1136/bmj.n2923 pmid:34893477
- 30. Geoghegan S, O’Callaghan KP, Offit PA. Vaccine Safety: Myths and Misinformation. Front Microbiol. 2020 Mar 17;11:372. pmid:32256465
- 31. Wawrzuta D, Klejdysz J, Jaworski M, Gotlib J, Panczyk M. Attitudes toward COVID-19 Vaccination on Social Media: A Cross-Platform Analysis. Vaccines (Basel). 2022 Jul 27;10(8):1190. pmid:35893839
- 32. Stellefson M, Paige SR, Chaney BH, Chaney JD. Evolving Role of Social Media in Health Promotion: Updated Responsibilities for Health Education Specialists. Int J Environ Res Public Health. 2020 Feb 12;17(4):1153. pmid:32059561
- 33. Mitgang EA, Blaya JA, Chopra M. Digital Health in Response to COVID-19 in Low- and Middle-income Countries: Opportunities and Challenges. Glob Policy. 2021 Jul;12(Suppl 6):107–109. Epub 2021 Mar 17. pmid:34230840