Core component.

The core component of ADEPT is a vaccine recommendation by prenatal providers to first-time PIs at the end of the second trimester (26–32 weeks), encouraging vaccinations during pregnancy and for their child after birth. Prior to intervention implementation, providers at intervention sites will complete four training sessions on the following topics: overview of the routine vaccination program in the US (including COVID-19 vaccine and RSV immunization recommendations), vaccine clinical trials and safety, and strategies for effective communication (2 sessions). During active implementation, prenatal providers will make recommendations encouraging timely maternal and childhood vaccinations.

Adaptive components.

The ADEPT intervention consists of two adaptive components: a multimedia vaccine education website and phone consultations with a vaccine navigator. The assignment of the adaptive component is determined based on a screening questionnaire that will assess vaccination intention, that is administered after the core component is delivered to the PI. PIs who have a positive vaccination intention will not receive any adaptive intervention components. PIs who are in favor of vaccinations but exhibit some degree of hesitancy (e.g., plan to delay some vaccines) will be offered access to the multimedia vaccine education website. PIs with negative intention to vaccinate or those unsure of their decision will be offered access to the multimedia vaccine education website, as well as a phone consultation with a vaccine navigator.

• Multimedia vaccine education website: An educational website was co-developed with parents. The website includes the following sections: About Us, About Vaccines, Vaccine Preventable Diseases, About the Schedule, Vaccination Schedule, Vaccines During Pregnancy, Visit Detail, What Doctors Say, Research with Confidence, Vaccines by Numbers, and More Resources. The ‘More Resources’ tab includes links to other credible websites such as cdc.gov and the American Academy of Pediatrics’ Healthy Children webpage for those interested in learning more. PIs without sufficient access to the internet or an internet-enabled device, may request access to a handout with the website content. Such individuals will also be reassigned automatically to receive a phone consultation with a vaccine navigator.
• Vaccine Navigator: The navigators are members of the study team and will connect with eligible PIs via phone. The vaccine navigators will participate in the provider trainings described with the core component of the intervention. The purpose of the navigator is to respond to questions or concerns not addressed by the website or during provider recommendation.

Cluster-Randomized Trial (CRT).

We will cluster-randomize 14 study practices in a 1:1 ratio to the intervention or control arm utilizing a covariate constrained randomization approach [23]. Practices in the intervention arm will implement the standard of care plus ADEPT whereas practices in the control arm will continue to implement the standard of care (no ADEPT, Fig 3). After the PIs in the intervention and control arms give birth, we will assess timeliness of the 2-month vaccines for the child using data from the state immunization registry. The primary study outcome will be the difference between the intervention and control arms in the timeliness of childhood vaccinations recommended for children at 2 months (see Outcomes).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Practice and participant flow between the cluster randomized trial and nested mixed methods study.

https://doi.org/10.1371/journal.pone.0313742.g003

Nested Mixed Methods (NMM).

The NMM study will follow an explanatory sequential design and include those PIs who are eligible to receive the adaptive intervention component(s) and who consent to research activities to provide feedback on the intervention. The quantitative component of the NMM study will consist of survey questionnaires, and the qualitative component will consist of in-depth interviews. The qualitative portion will occur after the quantitative data collection to help explain or elaborate on the quantitative findings. Specifically, the qualitative data will explore reasons for observed intervention effects and identify factors that can explain significant or non-significant findings. Using joint displays and narrative summaries, we will integrate qualitative and quantitative outcomes to explain any observed intervention effect [24].

Prenatal practices.

Prenatal care practices will be recruited using existing networks of the study investigators and members of the study external advisory board. Recruitment information will include a link to an interest survey. Any practices that indicate an interest in study participation will be contacted by study staff. If eligible, additional information will be collected from the selected practices for covariate constrained cluster randomization. In addition, to facilitate intervention implementation, each prenatal care practice will be asked to nominate a site champion for participation in the overall Study Advisory Board (SAB). The SAB will meet with study staff during the active implementation period. During these virtual meetings, suggestions or concerns may be discussed including recruitment strategy, recruitment and retention statistics, implementation challenges and solutions, and other lessons learned. To resolve any practice-specific challenges, study staff will meet with the site champions in one-on-one meetings.

Prenatal providers.

All prenatal providers and staff at prenatal practices randomized to the intervention arm will be eligible to participate in the provider training. For those individuals who provide informed consent, we will collect information on their characteristics (e.g., demographics, prior training, years in practice) and their feedback on training sessions. A subset of the providers will be invited to participate in in-depth interviews to discuss their experience making childhood vaccine recommendations to PIs.

PIs.

We will seek a waiver of consent for the assessment of the study outcomes among PIs and their children in the CRT since outcome data will be extracted from the state immunization registry where they are routinely reported, and any identifying information will be managed by an honest broker who is independent from the study team. Vaccination outcomes will be assessed at the practice-level and compared between study arms. Recruitment for the NMM study will occur in the intervention arm among those who self-identify as vaccine hesitant per the screening questionnaire. PIs who are eligible for the NMM study will be contacted by study staff and only those who provide informed consent will be enrolled in the NMM study. Participation in the NMM study is voluntary; PIs who do not want to participate will still have access to the ADEPT adaptive components as part of the CRT. Trained study staff will conduct the informed consent process in person or over the phone, with an electronic signature documenting consent. For the qualitative portion of the NMM study, PIs will be selected for the interview at the time of the endline survey (2 months post birth) to include perspectives from those who vaccinated their children on time, as well as those who delayed or refused vaccines at 2 months, according to CDC’s Birth-18 years immunization schedule (S4 File).

Provider trainings.

Quantitative and qualitative data will be collected as a part of the provider training. Post-training evaluation surveys will capture feedback on training sessions, including the format of training, and delivery. Evaluation surveys will be self-administered by the providers after each of the four training sessions. All providers who complete the trainings, regardless of participation in post-training evaluations, will be eligible for continuing medical education (CME) credits, including Maintenance of Certification (MOC) Part 4 credits from American Board of Obstetrics and Gynecology (ABOG) [25]. After the trainings have been completed, interviews will be conducted with a subset of 12 prenatal providers from intervention practices to assess any additional training needs and to better understand the experience of implementing vaccine recommendations in practice. These interviews may be completed in-person or virtually and will be conducted in English by a trained interviewer. Each interview is expected to last approximately 15 minutes and will be audio-recorded for transcription and analysis purposes.

CRT.

Representatives from each practice will complete a survey to provide data on the practice-level covariates necessary for constrained randomization and to assess readiness for implementation of ADEPT. At the time of outcome assessment, study practices will utilize medical records to generate a list of first-time PIs who were in their second and third trimester during the study period. The linkage of these data to the state immunization registry will be used to determine vaccination outcomes for children post-delivery. Identifiable data will be managed by an honest broker who will communicate with the state immunization registry for receiving vaccination data. De-identified datasets will be shared with the study team by the honest broker for calculation of vaccination outcomes.

NMM.

For the NMM study, each participant will complete the screening questionnaire to determine study eligibility, followed by a baseline survey, a midline survey (prior to birth of child), and an endline survey. Survey domains will include intent to vaccinate self and child, vaccine confidence (Emory Vaccine Confidence Index [26]), health information seeking patterns, and demographics. Midline and endline surveys will evaluate acceptability of the adaptive components of the intervention. Prior to the surveys, each participant will complete an informed consent form, which will describe study processes, risks and benefits. Qualitative data will be collected using open-ended questions and prompts in individual in-depth interviews from participants who completed the endline surveys. Interview question domains may include reasons for delayed or timely vaccinations, feedback on intervention components, and residual information needs. Interviews will be held in English, over the phone or in person and are expected to last approximately 30 minutes each. The interview guide themes and questions will be influenced by the quantitative outcomes and used to benefit from further exploration. Interviews will be audio recorded to facilitate data transcription and analysis.

Sample sizes.

Provider trainings. Only individuals from the intervention practices will be offered access to the trainings. All providers, staff and trainees at these practices will be eligible to participate in the training.

CRT. We anticipate the proportion of children who receive delayed or no vaccination at 2 months of age at baseline to be 20% [6]. We estimate that the intervention will reduce this proportion to 10%. We used a sample size formula for a two-sided Z-test comparing two proportions that accounts for the CRT design and incorporates a small-sample bias correction suitable for CRTs with fewer than 30–40 clusters in total (see equation (3) of Thomson et al, and more details in S5 File) [27]. With a sample size of 7 practices in each arm (14 total) and with at least 55 PIs/practice (770 total) the study would be powered at 90% to detect a difference in the proportion of 10% between arms at a two-sided alpha level of 5%, assuming that the coefficient of variation of cluster (i.e., practice-level) proportions with the primary outcome is 0.15 [27]. The target sample size is further inflated by 10% (total: 850) to accommodate a small reduction in power anticipated due to variable enrollment by practice and a small proportion of children with missing effectiveness outcomes due to out-of-state movement [28, 29]. In reality, the sample size for each practice will depend on practice size and eligibility of the PI population seen during the study period. Given that the primary outcome will be measured via data linkage of practice-level data to the state immunization registry using an honest broker, we will request an extract of data for all eligible PIs who are seen at the practice during the study period with the goal that the majority of enrolled practices would be expected to meet this target. Sensitivity analyses show that 80% power would be achieved if the proportion of children who delayed or refused vaccination at 2 months of age is 30% (rather than 20%) with a slightly smaller CV of 0.125 or even if the coefficient of variation is as large as 0.2 (S5 File).

NMM. Sample size for the quantitative surveys will be estimated with the assumption that 20% of 425 PIs in the intervention arm are vaccine hesitant, and, hence, 85 PIs will be eligible to receive the adaptive intervention components as part of the NMM study. Of these, 68 are expected to agree to participate and to have midline survey data available (accounting for loss to follow-up) and estimate that the intervention will reduce the proportion of vaccine-hesitant to 80% at that time point. Using exact binomial methods, data from 68 PIs in a non-clustered setting would provide a 95% confidence interval ranging from 68% to 89% for a proportion vaccine-hesitant of 80% with a small increase due to the small design effect anticipated due to clustering. For the qualitative portion, existing research suggests that thematic saturation can be achieved with 12 interviews, with meta-themes presenting as early as six interviews [30, 31]. The proposed sample size of 24 PIs should be sufficient to achieve thematic saturation of reasons for delayed/refused vaccinations (n = 12) or timely vaccinations (n = 12).

Provider training.

Summary characteristics for consenting providers will be described by study practice. Summary statistics will be calculated from data on provider training. Broad thematic domains from interviews with providers will be identified and cross referenced with quantitative data.

CRT.

Summary characteristics for study practices will be described by intervention and control arm. We will also describe the distribution of vaccination intention among PIs. A timeframe of 28 days from the vaccination due date will be used as the window of timeliness. Timely vaccination doses will receive a score of 2, doses that are delayed beyond the 28-day window will receive a score of 1, and the non-receipt of a dose during outcome assessment will be scored a 0. For the time point of outcome determination (2 months), we will create a cumulative timeliness score and use that to determine the proportion of children who receive all recommended vaccines in a timely fashion. For the analysis of the primary outcome of vaccination timeliness, we will use data on vaccination outcomes for children of all PIs enrolled in the study (including those who screened as vaccine-hesitant in the intervention arm). The generalized estimating equations (GEE) approach will be used to account for clustering of outcomes due to the hierarchical structure of data (PIs nested within practice). The child-level binary outcome (timely vs. not) will be regressed on an intervention-arm indicator, strata and the practice-level covariates used in the covariate constrained randomization procedure. We note that no additional adjustment by individual-level characteristics is possible due to the source of data, i.e., state immunization registry. Log and identity links will be used to obtain both relative and absolute intervention effects, as per recommendations from the CONSORT statement on trial reporting [22]. Robust standard errors will be used together with finite-sample corrections to reduce inflated Type I error rate, which is expected due to the small numbers of clusters (I.e. 14 practices) [33]. Given that outcomes are obtained from the state immunization registry, we do not expect missing data in the usual sense. Instead, some outcome data will not be available for those children whose birthing parents were enrolled towards the end of the study. Since this is by design, the missing data is not expected to bias the estimated intervention effect. A similar GEE approach will be used to model vaccination coverage.

NMM.

Summary characteristics of PIs enrolled and receiving the adaptive components of the intervention will be computed. The secondary outcome of the study, change in vaccine hesitancy from baseline to midline, will be computed utilizing data collected from enrolled PIs prior to receipt of the intervention, and at midline (up to 2 months post intervention, prior to birth). Given the anticipated sample size of PIs enrolled in the NMM study, we anticipate being underpowered to detect any meaningful pre-post difference. However, the determination of this outcome is important to inform sample size estimates for future studies.

Thematic analysis of qualitative data will occur using a phenomenological lens using the following steps: A) an initial read through of the transcripts to evaluate content and quality, b) coding of a priori themes based on the interview guide during a second read through and an additional review of 10% sample coded transcripts by a secondary team member, c) display of data through patterns and subthemes/codes and descriptions in thematic memos, and D) data reduction by applying matrices to organize and extract the key concepts on vaccine hesitancy.

Mixed methods data integration will occur using a joint display. Qualitative themes explaining the vaccination decision and describing residual barriers will be organized by a joint display by quantitative outcomes as follows: ADEPT and timely vaccinations, ADEPT, and delayed vaccinations. The joint display will be used to facilitate an explanation of the impact of ADEPT on timely vaccination [24].