https://orcid.org/0000-0002-8198-0826
Figures
Abstract
Adult vaccine-preventable infectious diseases contribute substantial burden each year, and vaccine uptake remains suboptimal. As digital health communication grows, digital newsletters may represent a scalable, low-cost tool to promote vaccination. We conducted a randomized prospective study to evaluate the impact of digital newsletters on subscribers’ knowledge, attitudes, and behaviors regarding four adult vaccines: respiratory syncytial virus (RSV), influenza, shingles, and COVID-19. Between November 2023 and January 2024, half of subscribers to the Those Nerdy Girls (TNG) Substack newsletter received additional vaccine-focused newsletters, while the other half received only the standard twice-weekly newsletters. Pre- and post-intervention online surveys assessed vaccine knowledge, attitudes, and behaviors among adult subscribers who consented to participate. Across 1,327 pre-intervention and 1,208 post-intervention participants, knowledge gains were observed for RSV and shingles vaccines in the intervention group compared with controls, while knowledge of influenza and COVID-19 vaccines did not improve. Attitudes toward vaccination were generally positive at baseline and showed no significant intervention effects across vaccine types. Likelihood of vaccination increased for influenza, COVID-19, and RSV overall during the study period, but only influenza vaccination showed a significantly greater increase in the intervention group relative to controls. No significant effects were found for shingles vaccination. These results indicated that digital newsletters can improve knowledge and, in some contexts, support uptake, but knowledge gains did not consistently translate into changes in attitudes or behaviors. Findings highlighted the limitations of knowledge-deficit models for adult vaccination promotion. Digital newsletters should be considered as one component of broader public health strategies, particularly when paired with approaches that address logistical access, trust, and social norms.
Citation: Jones M, Reilly M, Simanek A, Stewart A, Seshadri R, Dowd JB, et al. (2026) A randomized assessment of the impact of ‘Those Nerdy Girls’ newsletters on adult vaccination outcomes. PLoS One 21(3): e0344258. https://doi.org/10.1371/journal.pone.0344258
Editor: Nicole E. Basta, McGill University, CANADA
Received: October 14, 2025; Accepted: February 17, 2026; Published: March 12, 2026
Copyright: © 2026 Jones et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: A replication dataset can be found in the Open ICPSR data repository: Jones, Malia. TNG PRIA 2023 - Replication Dataset. Ann Arbor, MI: Inter-university Consortium for Political and Social Research [distributor], 2026-01-30. https://doi.org/10.3886/E244768V1.
Funding: This work was supported by an award to MJ from the National Institute of Allergy and Infectious Diseases (K01AI132741) https://www.niaid.nih.gov/. This work was supported by an award to JBD from the Leverhulme Trust (Large Center Grant) https://www.leverhulme.ac.uk/. These funders were not involved in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
At least 50,000 U.S. adults are estimated to die each year of vaccine-preventable diseases, with tens of thousands more experiencing hospitalization and millions more who become ill [1]. Common, vaccine-preventable adult illnesses include influenza, COVID-19, respiratory syncytial virus (RSV), and shingles [2,3]. Yet, adult vaccine uptake remains relatively low [3]. The Centers for Disease Control and Prevention (CDC) estimated that there were 9–41 million influenza cases, 120,000–710,000 hospitalizations, and between 6,300–52,000 deaths annually between 2010–2024 [4]. Despite influenza’s burden, from 2010–2024, only 37.1 to 50.2% of U.S. adults received an annual influenza vaccine [5].
Uptake among U.S. adults has been even lower for vaccines that have more recently become available. While 34,000−53,000 deaths and 290,000−450,000 hospitalizations from COVID-19 occurred from October 2024-July-2025 [6], only 23% of U.S. adults received the updated 2024−25 COVID-19 vaccine [7]. Indeed, influenza vaccine rates for adults have declined since the onset of the COVID-19 pandemic [5], in part due to spillover of COVID-19 vaccine hesitancy [8].
On July 21 2023, the Centers for Disease Control and Prevention (CDC) Advisory Committee on Immunization Practices (ACIP) recommended that people aged 60 years and older may receive a single dose of a vaccine against RSV, based on shared clinical decision-making [9]. This recommendation was revised the following year (after the study described here was concluded) [10]. Approximately 2–10% of older adults in the U.S. get RSV each year, leading to an estimated 11,000 annual deaths [11]. While one dose of the adult RSV vaccine is 75% effective at preventing RSV-related hospitalization [12], only 47.5% of U.S. adults aged 75 years and older received this vaccine as of April 2025 [7].
Beginning in 2017, ACIP recommended two doses of the shingles vaccine for all adults aged at least 50 years, and those aged 19 years and older with impaired immune systems [13]. Current estimates suggest that 1 in 3 people will develop shingles in their lifetime, and 10–18% of those who do will develop long-term nerve pain (i.e., postherpetic neuralgia) [14]. While two doses of shingles vaccine is 90% effective at preventing shingles and postherpetic neuralgia, as of 2018 fewer than 35% of U.S. adults aged 60 years and older were vaccinated [15].
In an increasingly digital health information environment [16], a growing number of interventions have focused on the rapidly evolving role that digital communication tactics may play in improving vaccination uptake among adults [17]. Such tactics include reminder phone calls, texts, and emails, often from clinicians or government health agencies; informational emails from official sources (such as clinicians or health systems); and health information from informal sources using modalities such as subscription-based digital newsletters, blogs, and social media accounts [16,18].
Automated digital reminders
Automated digital reminders in health communications refer to prompts delivered via digital channels (e.g., email, SMS/text messages, app notifications) that cue individuals to perform a health-related behavior [19]. These interventions attempt to nudge users toward desired health actions outside of direct clinical encounters [19]. Such digital reminders have been leveraged to cue those who are vaccine-eligible to get vaccinated [20,21]. However, evidence of the effectiveness of such reminders on increasing vaccine uptake has been mixed [22,23]. One recent systematic review focused on older adults found that, across digital reminder modalities (e.g., emails, texts, etc.), more than half of reviewed studies reported some evidence of increased vaccination uptake, though effects were heterogenous and often modest [24]. Another systematic review concluded that email reminders about vaccination were generally more effective than no intervention, but tended to be less effective or no more effective than other automated reminder modalities (e.g., texts, app notifications, phone calls), with variation across populations and study designs [25].
Informational communications
Digital reminders typically contain only a prompt to action, not information or educational content (e.g., descriptions of diseases, vaccines, eligibility criteria, or benefits and risks). Some recent vaccine promotion efforts have also focused on sending emails containing vaccine-related information. Such efforts are consistent with the knowledge-attitude-behavior theory of change, which suggests that providing information on vaccines will translate to changes in attitudes and subsequently vaccine behavior, given the information comes from trusted sources [26]. This framework aligns with the Health Belief Model, in which increasing knowledge can influence perceived key determinants of individuals’ motivation to engage in preventive health behaviors like vaccination, such as susceptibility, severity, benefits, and barriers [27].
A cluster-randomized trial conducted in Denmark directly compared the effect of reminders and informational content, both sent via email, from the Danish Health Authority. The informational email contained facts on influenza and COVID-19 vaccines [28]. The authors found that, compared to the simple reminder, repeated informational emails sent 14 days apart were associated with greater influenza vaccine uptake [28]. These findings suggest that email-based dissemination of vaccine-related information by a government health authority has a positive impact on influenza vaccine uptake among adults. Questions remain, however, regarding the impact of informational digital content on uptake of other adult vaccines.
People are also increasingly seeking health information online from less formal health authorities, such as content from social media platforms, YouTube creators, digital newsletters, and blogs [29]. In many of these modalities, users self-select or “subscribe” to content that interests them. Little is known about how vaccine-related information disseminated from such sources may affect changes in vaccine knowledge, attitudes, and behaviors [30,31]. One recent study assessed the effect of health information content shared a professional science communicator across a variety of digital media modes on adult influenza vaccine knowledge, attitudes, and behaviors [32]. Content in this study included digital newsletters delivered via a self-subscribing system, as well as podcasts, social media posts, and infographics. Results from cross-sectional surveys administered before and after the campaign showed that 4.7% of post-campaign respondents reported that they changed their minds about getting vaccinated for flu; a little more than half of those said the change was in part because of the targeted educational content of the campaign [32]. Given that the informational content was shared across a range of digital platforms and mediums, it is challenging to discern, however, which aspect(s) of the campaign most influenced the changes observed. In addition, only the adult influenza vaccine was studied. Taken together, there is a need to further assess the impact of digital communications that include informational content beyond vaccine reminders and are sent from entities other than clinicians and government agencies, on vaccine knowledge, attitudes, and behaviors for a wider array of adult vaccines. Our intervention is situated at the intersection of these emerging communication strategies.
Prospective randomized impact assessment of informational digital newsletter content on adult vaccine behaviors
Those Nerdy Girls (TNG), a collective of women and nonbinary scientists and clinicians, have provided practical, fact-based health information via multiple social media platforms since March 2020 and Substack digital newsletters since July 2021 [33–37]. Substack is a popular digital newsletter aggregator and delivery service that allows users to self-subscribe to content that interests them. Content on Substack is delivered to subscribers’ email inboxes and can also be viewed on the Substack app or website. Substack also suggests content that may be of interest to users based on its internal algorithms. TNG’s content across all platforms (including the Substack newsletter) includes practical information about COVID-19 and other infectious disease prevention, pediatric and adult vaccinations, reproductive health, mental health, aging, and data literacy.
In this study, we used a prospective, randomized design to measure the impact of an informational digital newsletter, delivered via Substack to an existing subscriber list, on vaccine knowledge, attitudes, and behaviors for four adult vaccines: RSV, influenza, shingles, and the Fall 2023 season COVID-19 vaccine. Between November 2023 and January 2024, half of subscribers to the TNG Substack newsletter received additional vaccine-focused newsletters, while the other half received only the standard twice-weekly newsletters. We hypothesized that following receipt of additional digital newsletters, vaccination knowledge, attitudes, and vaccine uptake for these four vaccines would increase more in the intervention group compared to the control group.
Methods
Intervention
At the time of the study launch (October 23, 2023), TNG’s free Substack newsletter had 12,721 subscribers. TNG typically sends its Substack subscribers twice-weekly newsletters, which are delivered via email and/or in the Substack app. We randomly assigned all TNG Substack newsletter subscribers to one of two groups. Half of the subscribers were assigned to a control group and received TNG’s regularly scheduled newsletters twice a week throughout the study. The other half were assigned to the intervention group, which received the same two newsletters plus two additional TNG newsletters each week featuring additional content about four adult vaccinations: RSV, shingles, influenza, and the Fall 2023 COVID-19 vaccine. A timeline of our study procedures is shown in Fig 1.
In total, intervention group subscribers received an additional 24 digital newsletters between November 1, 2023 and January 31, 2024. Each newsletter focused on a single vaccine. Content was distributed such that each of the four vaccines was featured in six separate newsletters over the intervention period (4 vaccines x 6 newsletters = 24 total). Because the intervention occurred during influenza vaccination season, which is relatively brief, content on influenza vaccine was concentrated during the first month of the study period (beginning Nov 1, 2023), and content on the other vaccines was distributed across the remaining 8 weeks. The full content, timing, and open rates of the individual intervention digital newsletters are available in S1 Appendix.. Example content is shown in Fig 2.
All digital newsletters followed the format of regular TNG Substack content. That is, they consisted of 300–600-word essays, and included a title and a leading short summary or Single Overarching Communication Objective [38] statement (examples in Fig 2). Intervention newsletters were written by TNG contributors with expertise in epidemiology, population health, and health policy. The intervention group newsletters provided information on topics including vaccine side effects, need for an updated vaccine where relevant, vaccine recommendation details (e.g., recommended ages and doses), vaccine efficacy, vaccine safety, morbidity and mortality from each virus, and cost of vaccines. Additional descriptive information about the newsletter topics is available in S1 Appendix.
Pre- and post-intervention survey and sample
Prior to the launch of intervention content (on Oct 23, 2023) and following the conclusion of the intervention (on February 15, 2024), all TNG Substack subscribers were invited via email to complete an online survey assessing knowledge, attitudes, and behaviors around the target vaccinations. Demographic information was also collected on either the pre- or post-intervention surveys, including gender, educational attainment, income quartile, household size, health insurance status, rural-urban self-identification, and political orientation (see response options in S2 Appendix). Age was collected at both times. The survey was fielded using Qualtrics software [39]. Eligible participants were aged at least 18 years and living in the U.S. TNG Substack newsletter subscribers received up to three email reminders to complete each survey. TNG also posted notices that each survey was open on its social media pages. There was no incentive to participate. The pre-intervention survey remained open for one week, and the post-intervention survey was open for two weeks.
During the study period, we received 683 requests to unsubscribe from TNG content via the Substack platform. Unsubscribe requests were tracked and synced to the survey invitation list to prevent those who unsubscribed from being invited to complete the post-intervention survey. Unsubscribe requests were evenly distributed between the control and intervention groups. Unsubscribe requests during the study period were also similar to the typical rate of unsubscribe requests TNG receives, about 1.5% per month.
At both pre- and post-intervention phases, all participants answered questions about their knowledge of all four vaccinations (whether they were age-eligible for the vaccination or not). In addition, each participant answered questions about their attitudes and behaviors for two vaccinations, depending on their age and/or random assignment, using a workflow illustrated in Fig 3.
We used differentiation by age to assign attitude and behavior questions because age-specific recommendations for each vaccination varied. At the time of the study, all people aged at least 60 were recommended to discuss RSV vaccination with a provider [9]; all people at least 50 were recommended to receive a two-dose shingles vaccination every 10 years [13]; and all adults were recommended to receive both influenza and COVID-19 vaccination each year [40]. Therefore, participants aged at least 60 years (at the time of each survey) received questions about RSV vaccination plus one of the other three vaccines, selected at random. Participants aged between 50 and 60 years received questions about shingles vaccination plus either the COVID-19 or influenza vaccine, selected at random. Participants aged younger than 50 years received questions about COVID-19 and influenza vaccination. Additional survey domains related to vaccination information-sharing behavior, diffusion of innovation, and subjective norms were also assessed but are not analyzed here. The full survey instrument is listed in S2 Appendix. The Minimum Risk IRB of the University of Wisconsin-Madison oversaw this study (protocol #2023−1332). Study participants gave written informed consent prior to participation.
Measures
Knowledge measures.
Study participants answered 6 knowledge questions about each vaccine. Response options included true, false, and “not sure.” For each vaccine, we created a vaccine knowledge index (i.e., count of incorrect answers; higher scores indicate lower knowledge) by summing the incorrect and “not sure” responses. Items were reverse coded as appropriate. To align with common reporting conventions, we computed Cronbach’s alpha separately for knowledge about each vaccine, which was low to moderate (influenza = 0.13, COVID-19 = 0.39, shingles = 0.46, RSV = 0.51). Because the knowledge assessments were designed as brief factual quizzes rather than psychometric scales intended to measure an underlying construct, internal consistency among items was not expected, and a low alpha is not necessarily indicative of poor measure quality in this context.
Attitude measures.
We based the attitude measures on the 5 C’s psychological antecedents of vaccine hesitancy [41], which include 5 subdomains of vaccine hesitancy: confidence, complacency, constraint, calculation, and collective responsibility. Each subdomain was assessed with multiple items, and participants rated their attitudes on a Likert scale, from strongly disagree (1) to strongly agree (5). There were 28 items assessing attitudes for shingles and RSV vaccines and 29 total items for influenza and COVID-19 vaccines. Items were reverse coded as appropriate. An overall vaccine attitude measure was constructed by calculating the mean across all items; higher scores indicate more positive attitudes. We computed Cronbach’s alpha for vaccine attitudes, which were high (influenza = 0.81, COVID-19 = 0.83, shingles = 0.82, and RSV = 0.77).
Behavior measures.
For each vaccine, the survey included questions assessing whether the respondent was up to date on the vaccine, and if not, whether they had intentions to get the vaccine in the next 30 days. These terms, and the questions used to assess them, varied slightly by vaccine to capture the nuances of vaccine recommendations (see S2 Appendix). For influenza and COVID-19 vaccines, we assessed whether the participant was up to date by asking if the participant had received a vaccine this season (example question text: “Since July 1st, 2023, have you received a flu vaccine? (That’s this season.)” Response options were “Yes,” “No,” and “Not sure.”) To assess whether participants were up to date on shingles vaccine recommendations, we asked if the participant had ever received any shingles vaccine. Those who answered yes were asked if it was a one-dose (Zostavax) vaccine or a two-dose (Shingrix) vaccine, and approximately when they had received it. We combined these responses analytically determine whether the participant was up to date with the current shingles vaccine recommendation. For RSV, the recommendation at the time was to talk to a provider about RSV vaccination. We therefore asked participants if they had talked to a clinician about RSV vaccination, what that recommendation was if so, and if they had received an RSV vaccine. In all cases, “not sure” answers were combined with “no” answers for analysis. Each behavioral outcome is measured with a single binary variable.
Analysis
All analyses were performed using Stata version 19 [42]. For analysis, we excluded all respondents who did not qualify for the study because they were not at least 18 years old (pre-intervention n = 4; post-intervention n = 1), were living outside the U.S. (pre-intervention n = 136, post-intervention n = 54), and those with less than 5% survey completion (pre-intervention n = 425, post-intervention n = 354).
Descriptive analysis.
We evaluated balance in demographic characteristics across intervention and control groups at pre- and post-intervention using Pearson chi-squared tests for independence. Demographic variables included age, gender, education level, income, household size, health insurance status, rural-urban classification, and political orientation.
Outcome models.
The analysis followed a pre-post design with a control group comparison. The main outcomes (vaccine knowledge, attitudes, and behaviors) were examined across four vaccine types (RSV, COVID-19, influenza, and shingles) using a regression modeling framework. For each outcome, we estimated models including independent variables for time (pre/post), group (intervention/control), and their interaction. The primary purpose of regression modeling is to assess the statistical significance of difference-in-differences. We used regression modeling approaches appropriate to the distribution of each outcome variable. Specifically, vaccine knowledge (count variables with strong right skewness) were modeled using Poisson regression. The vaccine attitude scales (continuous variables) were modeled using linear regression. For modeling, we transformed the vaccine attitude scale by z-standardizing to improve normality of its distribution. Behavior outcomes (binary variables) were analyzed using logistic regression.
To maintain consistency in analytic samples across knowledge, attitude, and behavioral outcomes for each vaccine, results for vaccine knowledge were restricted to the subset of participants who are age-eligible for the relevant vaccine. We reported results from unadjusted models. We conducted sensitivity analyses (not reported) in which we ran models for all outcomes adjusted for demographic variables (age, gender, educational attainment, income level, household size, political orientation, and rurality) to assess the potential impact of residual confounding on our results. Last, we also carried out subset analyses (not reported) in which we ran models limiting the analytic sample to repeat participants, using a repeated-measures (longitudinal) design.
Results
Sample characteristics
Open rates on intervention newsletters ranged from 48–59%, depending on the specific message (see S1 Appendix). Table 1 presents the number of participants by group and survey round. Completed survey response rates were approximately 10% in both the intervention and control groups, consistent across both pre-intervention and post-intervention. Demographic characteristics of the sample (Table 1) reflect the broader readership of TNG (i.e., across all platforms), based on metadata from TNG’s social media accounts (data not published). The modal respondent in this sample was a white woman aged 60–69 years with a graduate degree. Most participants reported progressive political views. Pearson’s chi-squared tests with a threshold of p < 0.05 indicated that all demographics were balanced across intervention and control groups at both pre- and post-intervention. Demographic characteristics were also not statistically different between the overall pre- and post-intervention samples.
Knowledge outcomes
Incorrect answers to knowledge questions were rare at pre-intervention across all vaccines, with a mean between 0.76 and 1.54 knowledge items incorrect (out of 6) (Table 2, Panel A). At post-intervention, participants got fewer knowledge items wrong across all vaccines (0.68–1.57 items incorrect).
Results of Poisson regression models with vaccine knowledge as the outcome of interest are shown in Table 3. The interaction terms in these models show the difference-in-differences (comparing pre-/post-intervention differences between the control and intervention groups). This term was statistically significant at p < 0.05 for RSV and shingles vaccine knowledge, demonstrating improved knowledge to a greater extent in the intervention group. Specifically, for RSV vaccine, the intervention respondents had a 28% greater decline in the rate of incorrect responses to knowledge questions relative to the control group (IRR of the interaction term = 0.72, CI 0.59–0.88). For shingles vaccine, the intervention group had a 16% greater decline in the rate of incorrect responses (IRR of the interaction term = 0.84, CI 0.71–0.98). However, there was no significant difference-in-difference for either influenza or COVID-19 vaccine knowledge.
Attitude outcomes
At the time of the pre-intervention survey, attitudes toward all four adult vaccinations were overall positive, with means between 4.05 and 5.30 (out of 6, before z-standardization), and were universally more positive at post-intervention, with means between 4.15 and 5.31. Results of linear regression modeling (Table 4) indicated that RSV attitude was statistically significantly higher in both groups at post-intervention, and although the pre-post difference was larger in the intervention group compared to the control group, this difference was not statistically significant at p < 0.05. For influenza, COVID-19, and shingles vaccines, attitude measures were not statistically significantly different from pre- to post-intervention surveys, nor when comparing pre-post differences between intervention and control groups.
Vaccination behaviors
At pre-intervention, vaccine coverage ranged from approximately 26% (for RSV, control group) to 75% (for influenza, control group) (Table 2, panel C). At post-intervention, vaccine rates were generally higher, ranging from 46% for RSV in the control group to 91% for influenza in the intervention group (Table 2, Panel C). At the time of the pre-intervention survey, 50% of participants had talked to a clinician about RSV vaccination. At post-intervention, 56–59% of participants had talked to a clinician about RSV vaccination (Table 2, Panel C).
Logistic regression for the behavioral outcomes (Table 5) indicated vaccination did increase statistically significantly from pre-to post-intervention for influenza, COVID-19, and RSV. Because these were unadjusted models, the predicted probabilities by group (intervention vs. control) and time (pre- vs. post-intervention survey) from this model were equivalent to the observed values reported in Table 2, panel C; modeling tested the significance of the interaction terms, shown in the “difference in differences” rows. Comparing the pre-post change across the control and intervention groups, we found a statistically significant difference in difference for influenza vaccination at p < 0.05. There was no statistically significant difference-in-difference for the other vaccination outcome measures. For influenza, there was a net 11 percentage point gain in vaccination uptake from pre- to post-intervention in the control group (i.e., 75% to 86%), and a net 22 percentage point gain in the intervention group (i.e., 69% to 91%), for a difference-in-difference of 12 percentage points.
In sensitivity analyses not shown, we ran models including controls for a range of demographic variables. Results had no notable differences in main effects, compared to the unadjusted models presented here. A subset analysis including only the repeat participants (also not shown) found main effects in the same direction as those reported in the main analysis. However, effects did not reach statistical significance.
Discussion
Taken together, this study tested the impact of a subscription-based digital newsletter on study participants’ knowledge, attitudes, and behaviors related to four adult vaccinations. Results differed by vaccine. For RSV and shingles vaccines, the intervention was associated with knowledge improvement without parallel changes in attitudes and behavior. For influenza vaccine, the intervention was associated with increased vaccine uptake despite no change in knowledge or attitudes. For COVID-19, no significant effects of the intervention were observed.
Previous studies show that digital reminders and informational content can increase vaccination [22–25,28,32]. We add to this literature by showing that digital newsletters delivered via email and the Substack app/website can influence knowledge, and in some cases uptake, among self-subscribed readers, but observed effects varied by vaccine.
The intervention’s differential effects suggest that the pathway from knowledge to behavior depends on vaccine type, audience, and context. For example, influenza vaccination behavior change may have been more responsive to the intervention than other vaccines, even in the absence of change in knowledge or attitudes. We are not able to directly assess why. It is possible that because influenza vaccine knowledge was very high at baseline in our study participants, it was not likely to have increased. It is also possible that because influenza vaccine is widely available, covered by insurance, and reinforced by consistent recommendations for years, additional knowledge was not required for behavior change to occur. Overall, our additional influenza vaccine content may have functioned more as a digital reminder than informational content. Finally, our influenza vaccine content was also concentrated in November, during influenza vaccination season [43]. The timing may have amplified behavior change. By contrast, RSV vaccination content was distributed over a span of 3 months. This vaccine was also new in Fall 2023, with the CDC recommending shared decision-making rather than universal uptake. Thus providers and the public were still learning about the vaccine, and insurance coverage was uncertain. These contextual factors may explain why knowledge improved, but attitude and behavior did not, for RSV vaccination.
The theory of change underlying many vaccine promotion efforts assumes a knowledge-attitude-behavior (KAB) progression. This framework, reflected in the Health Belief Model [44] and Theory of Planned Behavior [45], suggests that knowledge informs attitude, which in turn drives behavior. Our findings, however, challenge this assumption. Given the rapid changes in the health information context, including the rise of social media and other informal sources of health information (both correct and incorrect), the relevance of these older behavioral models may be changing.
Our findings illustrate that knowledge gains are not always sufficient, or even necessary, for behavior change, even in populations for whom major access barriers are limited. For RSV and shingles, knowledge improved without attitudinal or behavioral shifts. For influenza, behavior improved without measurable changes in knowledge or attitude. These findings support critiques of knowledge-deficit approaches [46–49] and highlight the importance of taking into account the functional, emotional, social, and structural determinants of vaccination, which may modify the KAB pathway. As Brewer et al. note, thoughts and feelings matter for vaccine uptake, but so do social norms and logistical supports (e.g., free access, workplace accommodations) [47]. Knowledge change is not without value, but may not be enough to produce improvements in vaccine behavior without additional support.
Limitations
We considered several limitations. Many effects were in the expected direction but did not reach statistical significance, suggesting the study may have been underpowered or that effects were small. Our pre-intervention survey results indicate that the study population was highly knowledgeable and had positive vaccine attitudes, so our small effects could indicate a ceiling effect on the amount of change possible in this group. The intervention “dose” or the time between intervention and follow-up may also have been insufficient, especially for newer vaccines like RSV that may require longer adoption windows or vaccines such as shingles for which there is no seasonal pattern. In addition, this study relied on repeated cross-sectional samples, not a true longitudinal panel, leaving open the possibility that differences in who responded were at play in our results. Sensitivity analyses conducted among repeat participants showed similar patterns but smaller, non-statistically significant effects, as would be expected with reduction in sample size and reduced variation due to repeated measures in a panel study design. Models adjusted for demographic factors were not substantively different in terms of main effects, supporting that the randomized assignment led to balanced assignment to control vs. intervention group. Finally, we examined multiple models, which raises the possibility of inflated Type I error due to multiple comparisons. However, the number of comparisons was limited, with analysis conducted across only 4–5 outcomes per construct (knowledge, attitude, behavior), reducing the likelihood that our findings are driven by chance alone.
It remains unclear whether changes stemmed from newsletter content specifically or from simply receiving more communications, whereby an increased number of newsletters could serve as a ‘nudge’ for behavior change. In addition, some of the intervention newsletters highlighted the timeliness and eligibility criteria for adult vaccinations, situating them more in the digital reminder space than the information space. However, average newsletter open rates were about the same across vaccines in the study (influenza 55%, COVID-19 55%, RSV 53%, shingles 54%), suggesting that differences were not due to a nudge effect alone. Also, because we did not systematically vary message framing or calls to action, we cannot identify which elements were most effective.
Finally, our sample was not representative of the general population: TNG readers broadly (i.e., across all platforms) generally have expressed high trust in its content [50]. Most participants were politically progressive, highly educated women, and had high vaccine knowledge and positive vaccine attitudes at baseline. In addition, our survey response proportion (about 10% with usable responses) could introduce respondent bias. These factors limit generalizability of our findings, though similar strategies may work in other trusted-audience contexts.
Despite these limitations, this study offers rare experimental evidence on digital newsletter interventions for improving adult vaccination uptake. Although scalability and cost were not evaluated in this study, digital newsletters represent a delivery modality that is widely used and low-cost, underscoring the value of further research to assess their effectiveness across diverse populations and contexts.
Conclusions
Subscriber-based digital newsletters are a low-cost, scalable, and timely communication tool that can be tailored to specific audiences. Our results suggest they can increase vaccine knowledge and behavior within engaged subscriber populations, but effects on uptake depended on vaccine type, and we speculate reasons for this variation included timing and historical context of each vaccine. Future research should test this approach in more diverse populations, vary content design (e.g., behavioral science-informed framing, calls to action), and compare digital newsletters with other communication strategies. Overall, knowledge gains did not consistently translate into behavior change, underscoring the need to pair information with strategies addressing social, emotional, and structural determinants of vaccination.
Supporting information
S1 Appendix. Digital newsletter content details.
https://doi.org/10.1371/journal.pone.0344258.s001
(PDF)
Acknowledgments
We thank Russell Dimond of the University of Wisconsin–Madison Social Science Computing Cooperative for helpful statistical consultation. We are also grateful to all the TNG contributors, especially Gretchen Peterson, who was instrumental in the logistics for this study. We are always grateful to our readers, particularly the ones who took the time to complete this way-too-long survey.
References
- 1. Doherty MT, Aris E, Servotte N, Beck E. Capturing the value of vaccination: impact of vaccine-preventable disease on hospitalization. Aging Clin Exp Res. 2022;34:1551–61.
- 2.
CDC. Vaccine-Preventable Adult Diseases. In: Vaccine Information for Adults [Internet]. 2025 [cited 14 Jan 2026]. Available from: https://www.cdc.gov/vaccines-adults/diseases/index.html
- 3.
The National Vaccine Program Office. National Adult Immunization Plan. US Department of Health and Human Services. 2019. Available from: https://www.hhs.gov/sites/default/files/nvpo/national-adult-immunization-plan/naip.pdf]
- 4.
CDC. About Estimated Flu Burden. In: Flu Burden [Internet]. 2024 [cited 13 Aug 2025]. Available from: https://www.cdc.gov/flu-burden/php/about/index.html
- 5.
CDC. Flu Vaccination Coverage, United States, 2023–24 Influenza Season. In: FluVaxView [Internet]. 2025 [cited 13 Aug 2025]. Available from: https://www.cdc.gov/fluvaxview/coverage-by-season/2023-2024.html
- 6.
CDC. Preliminary Estimates of COVID-19 Burden for 2024-2025. In: COVID-19 [Internet]. [cited 19 Jan 2026]. Available from: https://www.cdc.gov/covid/php/surveillance/burden-estimates.html
- 7.
CDC. Vaccination Trends. In: Respiratory Illnesses [Internet]. 2025 [cited 13 Aug 2025]. Available from: https://www.cdc.gov/respiratory-viruses/data/vaccination-trends.html
- 8. Zhang V, Zhu P, Wagner AL. Spillover of vaccine hesitancy into adult COVID-19 and influenza: the role of race, religion, and political affiliation in the United States. Int J Environ Res Public Health. 2023;20(4):3376. pmid:36834071
- 9. Melgar M, Britton A, Roper LE, Talbot HK, Long SS, Kotton CN, et al. Use of respiratory syncytial virus vaccines in older adults: recommendations of the advisory committee on immunization practices - United States, 2023. MMWR Morb Mortal Wkly Rep. 2023;72(29):793–801. pmid:37471262
- 10. Britton A, Roper LE, Kotton CN, Hutton DW, Fleming-Dutra KE, Godfrey M, et al. Use of respiratory syncytial virus vaccines in adults aged ≥60 years: updated recommendations of the advisory committee on immunization practices - United States, 2024. MMWR Morb Mortal Wkly Rep. 2024;73(32):696–702. pmid:39146277
- 11. Branche AR, Falsey AR. Respiratory syncytial virus infection in older adults: an under-recognized problem. Drugs Aging. 2015;32(4):261–9. pmid:25851217
- 12. Surie D, Self WH, Zhu Y, Yuengling KA, Johnson CA, Grijalva CG, et al. RSV vaccine effectiveness against hospitalization among US adults 60 years and older. JAMA. 2024;332:1105–7.
- 13. Anderson TC, Masters NB, Guo A, Shepersky L, Leidner AJ, Lee GM, et al. Use of Recombinant zoster vaccine in immunocompromised adults aged ≥19 years: recommendations of the advisory committee on immunization practices - United States, 2022. MMWR Morb Mortal Wkly Rep. 2022;71(3):80–4. pmid:35051134
- 14.
CDC. Shingles Facts and Stats. In: Shingles (Herpes Zoster) [Internet]. 2025 [cited 13 Aug 2025]. Available: https://www.cdc.gov/shingles/data-research/index.html
- 15. Terlizzi EP, Black LI. Shingles Vaccination Among Adults Aged 60 and Over: United States, 2018. NCHS Data Brief. 2020. pp. 370. Available: https://www.cdc.gov/nchs/products/databriefs/db370.htm
- 16. Stark AL, Geukes C, Dockweiler C. Digital health promotion and prevention in settings: scoping review. J Med Internet Res. 2022;24(1):e21063. pmid:35089140
- 17. Stockwell MS, Fiks AG. Utilizing health information technology to improve vaccine communication and coverage. Hum Vaccin Immunother. 2013;9(8):1802–11. pmid:23807361
- 18. Matangi N, Joachim-Célestin M, Granillo C, Rodarte V, Buckles B, Ashby T, et al. A mixed methods exploration of social media use for health information in under-resourced communities. Int J Environ Res Public Health. 2025;22(7):1081. pmid:40724148
- 19. Pirolli P, Mohan S, Venkatakrishnan A, Nelson L, Silva M, Springer A. Implementation intention and reminder effects on behavior change in a mobile health system: a predictive cognitive model. J Med Internet Res. 2017;19(11):e397. pmid:29191800
- 20. Dai H, Saccardo S, Han MA, Roh L, Raja N, Vangala S, et al. Behavioural nudges increase COVID-19 vaccinations. Nature. 2021;597:404–9.
- 21. Regan AK, Bloomfield L, Peters I, Effler PV. Randomized controlled trial of text message reminders for increasing influenza vaccination. Ann Fam Med. 2017;15(6):507–14. pmid:29133488
- 22. Zhang J, Jin H. Interventions involving nudge theory for COVID-19 vaccination: a systematic review and meta-analysis. Health Psychol. 2024;43(11):779–90. pmid:39052381
- 23. Reñosa MDC, Landicho J, Wachinger J, Dalglish SL, Bärnighausen K, Bärnighausen T, et al. Nudging toward vaccination: a systematic review. BMJ Glob Health. 2021;6(9):e006237. pmid:34593513
- 24. Buja A, Grotto G, Taha M, Cocchio S, Baldo V. Use of information and communication technology strategies to increase vaccination coverage in older adults: a systematic review. Vaccines (Basel). 2023;11(7):1274. pmid:37515089
- 25. Frascella B, Oradini-Alacreu A, Balzarini F, Signorelli C, Lopalco PL, Odone A. Effectiveness of email-based reminders to increase vaccine uptake: a systematic review. Vaccine. 2020;38(3):433–43. pmid:31806532
- 26. Bettinghaus EP. Health promotion and the knowledge-attitude-behavior continuum. Prev Med. 1986;15(5):475–91. pmid:3774779
- 27. Limbu YB, Gautam RK, Pham L. The health belief model applied to COVID-19 vaccine hesitancy: a systematic review. Vaccines. 2022;10:973.
- 28. Johansen ND, Vaduganathan M, Bhatt AS, Lee SG, Modin D, Claggett BL, et al. Electronic nudges to increase influenza vaccination uptake in Denmark: a nationwide, pragmatic, registry-based, randomised implementation trial. Lancet. 2023;401(10382):1103–14. pmid:36889332
- 29. Piltch-Loeb R, Silver D, Kim Y, Abramson D. COVID-19 vaccine information seeking patterns and vaccine hesitancy: a latent class analysis to inform practice. J Public Health Manag Pract. 2024;30:183.
- 30. Di Domenico G, Nunan D, Pitardi V. Marketplaces of misinformation: a study of how vaccine misinformation is legitimized on social media. J Public Policy Mark. 2022;41(4):319–35.
- 31. Wu Y, Kuru O, Kim DH, Kim S. COVID-19 news exposure and vaccinations: a moderated mediation of digital news literacy behavior and vaccine misperceptions. Int J Environ Res Public Health. 2023;20(1):891. pmid:36613213
- 32. Steier JB. Using social media to combat influenza vaccine misinformation and improve uptake: a social media campaign and repeated cross-sectional survey analysis. Mayo Clin Proc Digit Health. 2025;3(3):100229. pmid:40607109
- 33. Albrecht SS, Aronowitz SV, Buttenheim AM, Coles S, Dowd JB, Hale L, et al. Lessons learned from dear pandemic, a social media–based science communication project targeting the COVID-19 infodemic. Public Health Rep. 2022.
- 34. Golos AM, Guntuku SC, Piltch-Loeb R, Leininger LJ, Simanek AM, Kumar A, et al. Dear Pandemic: A topic modeling analysis of COVID-19 information needs among readers of an online science communication campaign. PLoS One. 2023;18(3):e0281773. pmid:36996093
- 35. Leininger LJ, Albrecht SS, Buttenheim A, Dowd JB, Ritter AZ, Simanek AM, et al. Fight like a nerdy girl: the dear pandemic playbook for combating health misinformation. Am J Health Promot. 2022;36(3):563–7. pmid:35164543
- 36.
Dear Pandemic: Nurses as key partners in fighting the COVID‐19 infodemic - Ritter. Public Health Nursing. Wiley Online Library; 2021. [cited 8 Aug 2024]. Available from: https://onlinelibrary.wiley.com/doi/abs/10.1111/phn.12903
- 37. Piltch-Loeb R, James R, Albrecht SS, Buttenheim AM, Dowd JB, Kumar A, et al. What Were the Information Voids? A qualitative analysis of questions asked by dear pandemic readers between August 2020-August 2021. J Health Commun. 2023;28(sup1):25–33. pmid:37390014
- 38.
Gamhewage G. Effective Communications Participant Handbook, World Health Organization. World Health Organization; 2015. Available from: https://www3.paho.org/hq/dmdocuments/2016/2016-cha-effective-comm-partic-handbook-who.pdf
- 39.
Qualtrics. Provo, UT. Available from: https://www.qualtrics.com
- 40. Regan JJ. Use of updated COVID-19 vaccines 2023–2024 formula for persons aged ≥6 months: recommendations of the Advisory Committee on Immunization Practices — United States, September 2023. MMWR Morb Mortal Wkly Rep. 2023;72.
- 41. Betsch C, Schmid P, Heinemeier D, Korn L, Holtmann C, Böhm R. Beyond confidence: Development of a measure assessing the 5C psychological antecedents of vaccination. PLoS One. 2018;13(12):e0208601. pmid:30532274
- 42.
StataCorp. Stata Statistical Software: Release 19. College Station, TX: StataCorp LLC.; 2025.
- 43.
Cleveland Clinic. When Is the Best Time To Get Your Flu Shot? In: Cleveland Clinic [Internet]. 2025 [cited 12 Jan 2026]. Available: https://health.clevelandclinic.org/when-to-get-flu-shot
- 44. Rosenstock IM. Historical origins of the health belief model. Health Educ Monogr. 1974;2(4):328–35.
- 45. Ajzen I. The theory of planned behavior. Organ Behav Hum Decis Process. 1991;50:179–211.
- 46. Dubé E, Laberge C, Guay M, Bramadat P, Roy R, Bettinger J. Vaccine hesitancy: an overview. Hum Vaccin Immunother. 2013;9(8):1763–73. pmid:23584253
- 47. Brewer NT, Chapman GB, Rothman AJ, Leask J, Kempe A. Increasing vaccination: putting psychological science into action. Psychol Sci Public Interest. 2017;18:149–207.
- 48. Nyhan B, Reifler J. Does correcting myths about the flu vaccine work? An experimental evaluation of the effects of corrective information. Vaccine. 2015;33(3):459–64. pmid:25499651
- 49. Kitta A, Goldberg DS. The significance of folklore for vaccine policy: discarding the deficit model. Crit Public Health. 2016;27(4):506–14.
- 50.
Jones M. Results from a reader impact assessment of Dear Pandemic. Oral Presentation presented at: Interdisciplinary Association of Population Health Scientists. Annapolis, MD; 2023.