https://orcid.org/0000-0002-7622-6562
Figures
Abstract
Background
COVID-19 vaccine coverage in the Latinx community depends on delivery systems that overcome barriers such as institutional distrust, misinformation, and access to care. We hypothesized that a community-centered vaccination strategy that included mobilization, vaccination, and “activation” components could successfully reach an underserved Latinx population, utilizing its social networks to boost vaccination coverage.
Methods
Our community-academic-public health partnership, “Unidos en Salud,” utilized a theory-informed approach to design our “Motivate, Vaccinate, and Activate” COVID-19 vaccination strategy. Our strategy’s design was guided by the PRECEDE Model and sought to address and overcome predisposing, enabling, and reinforcing barriers to COVID-19 vaccination faced by Latinx individuals in San Francisco. We evaluated our prototype outdoor, “neighborhood” vaccination program located in a central commercial and transport hub in the Mission District in San Francisco, using the Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) framework during a 16-week period from February 1, 2021 to May 19, 2021. Programmatic data, city-wide COVID-19 surveillance data, and a survey conducted between May 2, 2021 and May 19, 2021 among 997 vaccinated clients ≥16 years old were used in the evaluation.
Results
There were 20,792 COVID-19 vaccinations administered at the neighborhood site during the 16-week evaluation period. Vaccine recipients had a median age of 43 (IQR 32–56) years, 53.9% were male and 70.5% were Latinx, 14.1% white, 7.7% Asian, 2.4% Black, and 5.3% other. Latinx vaccinated clients were substantially more likely than non-Latinx clients to have an annual household income of less than $50,000 a year (76.1% vs. 33.5%), be a first-generation immigrant (60.2% vs. 30.1%), not have health insurance (47.3% vs. 16.0%), and not have access to primary care provider (62.4% vs. 36.2%). The most frequently reported reasons for choosing vaccination at the site were its neighborhood location (28.6%), easy and convenient scheduling (26.9%) and recommendation by someone they trusted (18.1%); approximately 99% reported having an overall positive experience, regardless of ethnicity. Notably, 58.3% of clients reported that they were able to get vaccinated earlier because of the neighborhood vaccination site, 98.4% of clients completed both vaccine doses, and 90.7% said that they were more likely to recommend COVID-19 vaccination to family and friends after their experience; these findings did not substantially differ according to ethnicity. There were 40.3% of vaccinated clients who said they still knew at least one unvaccinated person (64.6% knew ≥3). Among clients who received both vaccine doses (n = 729), 91.0% said that after their vaccination experience, they had personally reached out to at least one unvaccinated person they knew (61.6% reached out to ≥3) to recommend getting vaccinated; 83.0% of clients reported that one or more friends, and/or family members got vaccinated as a result of their outreach, including 18.9% who reported 6 or more persons got vaccinated as a result of their influence.
Conclusions
A multi-component, “Motivate, Vaccinate, and Activate” community-based strategy addressing barriers to COVID-19 vaccination for the Latinx population reached the intended population, and vaccinated individuals served as ambassadors to recruit other friends and family members to get vaccinated.
Citation: Marquez C, Kerkhoff AD, Naso J, Contreras MG, Castellanos Diaz E, Rojas S, et al. (2021) A multi-component, community-based strategy to facilitate COVID-19 vaccine uptake among Latinx populations: From theory to practice. PLoS ONE 16(9): e0257111. https://doi.org/10.1371/journal.pone.0257111
Editor: Jonathan Garcia, Oregon State University, UNITED STATES
Received: June 11, 2021; Accepted: August 23, 2021; Published: September 20, 2021
Copyright: © 2021 Marquez et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The work was funded by University of California, San Francisco; Chan Zuckerberg Initiative; San Francisco Department of Public Health; and A.D.K and L.R were funded by National Institutes of Health (T32 AI060530).
Funding: This work was supported by University of California, San Francisco (http://www.ucsf.edu), the Chan Zuckerberg Initiative (https://urldefense.com/v3/__https://chanzuckerberg.com__;!!LQC6Cpwp!_kOZyaVmp6cASxENUlQH2bpstcBdps63uocYYCdsMxaO5vd9CxrZckurtdxZ0dDkrqqPlw$), and the San Francisco Department of Public Health. A.D.K and L.R. was funded by T32 AI060530. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
COVID-19 has disproportionately affected underserved communities of color, including Latinx in the United States, further amplifying long-standing health disparities [1–4]. The highly safe and effective COVID-19 vaccines are the most critical tool in our public health strategy to overcome the COVID-19 pandemic. The success of our public health vaccination strategy relies on our ability to rapidly reach the population at highest risks of COVID-19, including communities of color which have been subject to decades of inequities and often have the least access to health care, including vaccination [5, 6].
There is a growing understanding of the barriers to vaccination communities of color face in the United States. In California where Latinx persons account for approximately 40% of the total population and 63% of COVID-19 cases to date, they have only received 27% of all COVID-19 vaccinations administered statewide [7, 8]. Some of the barriers to COVID-19 vaccine uptake among Latinx persons in the United States include a distrust in health systems stemming from historical experience, structural and medical racism [9], anti-immigration policies [10, 11] and anti-immigrant rhetoric [12], and inadequate access to language-concordant information on vaccinations [2, 13–15]. Latinx and other socio-economically vulnerable populations may also face substantial structural barriers to vaccine access, including less access to health insurance or a primary care provider, inadequate access to language/culturally-concordant health services, online appointment registration services that may be difficult to access and/or navigate, requirements to show identification and prove vaccine eligibility (possibly due to immigration concerns or difficulty obtaining official identification), and the direct and indirect costs associated with scheduling and attending vaccination appointments that may differentially impact low-income individuals [5, 16]. There are few formal evaluations of vaccination strategies that can overcome these barriers, and none for COVID-19 vaccines.
Social network interventions are increasingly used as part of public health strategies to increase the reach and uptake of evidence-based interventions [17]. They may also effectively overcome many of the barriers Latinx and other vulnerable populations face to COVID-19 vaccination. For example, persons who have been vaccinated against COVID-19 may serve as trusted and credible source for reliable vaccine-related knowledge, and positive vaccination experiences for their friends, family members and colleagues [18]. Further, vaccinated individuals can provide empathy and support their unvaccinated peers in getting vaccinated. Despite the large potential of social network interventions to help overcome medical mistrust as well as insufficient information and/or misinformation regarding COVID-19 vaccines by utilizing trusted peer messengers, to date there have been no published evaluations of social network-based interventions to improve COVID-19 vaccine coverage and uptake.
Community-academic partnership approaches can improve health equity by involving communities and their allies in the co-development of solutions to health problems that disproportionately impact them. Such partnerships facilitate the two-way exchange of knowledge and expertise between community members and academic researchers and also help ensure that interventions and implementation strategies are aligned with community needs [19]. Successful community-academic partnerships may result in the development of more feasible, acceptable and sustainable health strategies and may also increase trust among community members, all of which may facilitate increased uptake of evidence based-based interventions. In April 2020, we founded a community-academic-public health partnership called Unidos en Salud (“United in Health”, UeS) to respond to and support the Latinx community in the Mission neighborhood of San Francisco during California’s COVID-19 shelter-in-place orders [20]. Since then, UeS has provided ongoing, community-based, low-barrier SARS-CoV-2 testing, surveillance and support services [1, 20–23].
We hypothesized that a community-centered, culturally-tailored, theory-informed vaccination strategy that included mobilization, vaccination and “activation” components could successfully overcome key multi-level barriers to reach an underserved Latinx population, in part by utilizing its social networks to boost vaccination coverage. We developed this multifaceted approach via our community-academic-public health partnership. In this paper we describe the “Motivate, Vaccinate, and Activate” vaccination program and evaluate the program according to the Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) framework [24] during a 16-week period from February 1 and May 19th 2021.
Methods
Setting
The Unidos en Salud (UeS) neighborhood vaccination program was implemented in the Mission District, home to a large Latinx and immigrant community in San Francisco, California [25]. The Mission Neighborhood comprises a large majority of the 94110 zip code, which has an estimated population of 72,380 persons (62,452 ≥16 years old) of whom 33.4% identify as Latinx, 43.8% white, 14.7% Asian and 3.3% Black [26]. The neighborhood is economically heterogenous; the median household income is $134,592 per year, yet 22.6% of households have a combined income less than $50,000. The Mission District is an important cultural and commercial hub for Latinx people living throughout the many neighborhoods of San Francisco’s Southeast sector, which have consistently had the highest rates of COVID-19 throughout the pandemic (S1 Fig), concentrated among low-income, front line workers, unable to work from home [1, 23]. UeS has offered free walk-up, COVID-19 testing in the Mission District since April, 2020.
Ethics
The study was conducted under a public health surveillance program that was reviewed by the UCSF Committee on Human Research and determined to be exempt from IRB oversight. Survey participants provided written informed consent in their preferred language that was electronically documented prior to survey initiation. Parents or legal guardians provided written informed consent for minors under the age of 18.
Community-academic-public health partnership model
The Unidos en Salud community-academic-public health partnership founded in April 2020 to support the Latinx community in San Francisco during the COVID-19 pandemic and continues to provide ongoing community-based services. The partnership includes the San Francisco Latino Task Force-Response to COVID-19 (LTF), the University of California, San Francisco (UCSF), the University of California, Berkeley, the Chan Zuckerberg Biohub, locally owned Bay Area Phlebotomy and Laboratory Services (BayPLS), Primary.Health, and the San Francisco Department of Public Health (SFDPH). The LTF is a group made up of members and leaders from more than three dozen Latinx, community-based organizations, many of which are long-standing, that was forged during the COVID-19 pandemic [27]. Primary.Health informatics was founded in 2020 to meet community based COVID-19 testing efforts and provides cloud-based support for COVID-19 testing and vaccination registration and data metric tracking. BayPLS has bilingual staff that have provided community testing and vaccination services in the San Francisco Bay Area since the beginning of the COVID-19 pandemic. UeS operates via joint decision making by leaders from each of the partners and academic institution faculty that occur at weekly meetings. Funding for the vaccination program was provided by a combination of the SFDPH, UCSF, private donors and the Chan-Zuckerberg Initiative.
Overview and design of a strategy to reach and increase COVID-19 vaccination among Latinx individuals
For our vaccine strategy prototype, we utilized a theory-informed approach to design a multicomponent, implementation strategy that addressed barriers to COVID-19 vaccination faced by Latinx and other community members (Table 1). We specifically sought to reach those community members for whom the City’s high volume vaccination sites posed barriers such as a lack of transportation and “institutional mistrust.” We built upon lessons learned providing community-based COVID-19 services to socioeconomically vulnerable individuals [13–15], including providing free walk-up SARS-CoV-2 testing, with support for persons testing positive, to over 30,000 persons since April 2020. Formative findings from this work were used to design our “Motivate, Vaccinate, and Activate” strategy (Fig 1). The “Motivate, Vaccinate, and Activate” strategy’s design was guided by the PRECEDE Model [28] and therefore sought to address and overcome predisposing, enabling and reinforcing barriers to COVID-19 vaccination faced by Latinx and other low-income community members in San Francisco (Table 1). The “Motivate, Vaccinate, and Activate” strategy represents a culturally-tailored, multicomponent implementation strategy to optimize reach and uptake of COVID-19 vaccination among Latinx individuals in San Francisco. Further details for each strategy component are provided below and are summarized in Table 1.
“Motivate, Vaccinate, and Activate” strategy components
Community mobilization and demand generation activities (“Motivate”).
We used several methods to inform community members about COVID-19 vaccination and its benefits, raise awareness about COVID-19 vaccine eligibility and the UeS neighborhood vaccination site, and ultimately increase motivation and demand for COVID-19 vaccination (Table 1). Responses to a survey on vaccine attitudes and preferences from community members seeking COVID-19 testing at our site in January 2021 were used to inform training of our community workers and informational materials on vaccination [13]. Vaccine efficacy, short- and long-term side effects of vaccines, and conspiracy theories on motivation behind vaccine development were some of the key topics of concern that were directly addressed through educational and outreach efforts.
UeS community workers performed direct community outreach in the Mission District, via door-to-door household canvassing with flyers and by speaking to business owners in the commercial corridor of the Mission. We also emphasized to elder care facilities the opportunity and rationale to vaccinate high-risk adults. The LTF reached out to their multiple CBO’s and network of community organization to push out invitations to priority groups such as community health workers as they became eligible; this guaranteed that we were reaching our key populations who didn’t necessarily have access to the mass media advertising of vaccination appointments. Additionally, automated text messages (and reminder texts) were sent to 26,206 unique phone numbers of community members who had previously been tested for COVID-19 at a UeS site as soon as they became eligible for vaccination and invited them to get vaccinated at the UeS neighborhood site. Furthermore, flyers and posters were posted throughout the community (Fig 2a), and UeS members undertook Spanish radio, newspaper and television interviews to feature the UeS neighborhood vaccination site. Community leaders vaccinated at the site posted photos on social media (Facebook, TikTok), encouraging others to get vaccinated. Additionally, to engage community members who trust physicians but do not have access to one, we hosted Sunday ‘cafecitos’ directly across from the vaccination site, where community members could walk up and ask Latinx physicians questions while enjoying free pan dulce and coffee.
Panel A shows an example of flyers promoting COVID-19 vaccination at the Unidos en Salud neighborhood site, posted in English and Spanish. Panel B shows the registration area of the Unidos en Salud low-barrier vaccination site. Panel C shows the Unidos en Salud COVID-19 testing site directly across from the vaccination site. Community members could make vaccination appointments and were also encouraged to drop-in for same-day vaccination. Community members could ask site staff questions about the COVID-19 vaccine and were provided handouts about the COVID-19 vaccine. Panel D shows the waiting area of the Unidos en Salud low-barrier vaccination site where clients were provided education on possible COVID-19 side effects and were also ‘activated’ to the community health team be become vaccine ambassadors to reach out to their unvaccinated family members, friends and co-workers.
A client-centered neighborhood vaccination site (“Vaccinate”).
Neighborhood vaccination site characteristics. The UeS neighborhood vaccination site was the first of 8 community-led sites established in partnership with the SFDPH to increase equitable distribution of vaccines to neighborhoods disproportionately affected by COVID-19. The site was located outdoors and was located in a small parking lot (approximately 8,800 square feet), behind a McDonald’s restaurant at 24th and Capp Street (Fig 2b). The vaccination site was across the street from the free, walk-up UeS COVID-19 testing site, located at a busy public plaza and transport hub—the intersection of both above ground bus/streetcar system (MUNI) and underground subway system (BART) (Fig 2c). This location was intentionally chosen in order to enhance visibility and promote walk-up scheduling as people exited public transit and walked through the neighborhood. The site was open 4 days a week (Sunday through Wednesday) between the hours of 9am and 4pm. It was designed to be open in order to improve flow, and had several semi-permanent tents corresponding to different aspects of the vaccination process (i.e., check-in, pre-vaccination waiting area, vaccination area, and post-vaccination waiting area) (Fig 2d); the tents also provided privacy from the busy surrounding area as well as protection from the elements. The site played music in order to attract persons passing by and to create a welcoming and positive environment for those attending the site. Large, colorful signs in both Spanish and English were hung around and near the site to generate further awareness and encourage community members to register to get vaccinated.
The vaccine site officially opened on February 1, 2021 and remains operational. It provided clients either the Pfizer or Moderna COVID-19 mRNA two-dose vaccine, depending on availability. In order to minimize inconvenience for busy, socioeconomically vulnerable community members, the vaccine strategy prioritized smooth logistics and avoiding any need to reschedule appointments. To this end, while following dosage segment regulations we worked closely with the SFPDH and California health officials to mitigate any disruptions in vaccine availability. We further limited vaccination appointments to reduce the likelihood of vaccine stock-out and to facilitate site logistics. As we developed clinic operating protocols and improved efficiency, we were able to raise the appointment cap to 500 per day.
Vaccination site personnel and customer service principles (client-centeredness). The UeS neighborhood vaccination site was predominantly staffed by trained members of the local community who were bicultural and bilingual or monolingual Spanish speakers. Vaccinations were provided by bicultural and bilingual Spanish-speaking BayPLS and UeS staff, many whom had worked for prior UeS mass community-based COVID-19 testing events. The number of staff on site changed throughout the implementation period based on demand and ranged between 25 and 30 and peaked at 40 during mid-April 2021 when the general population became eligible for the COVID-19 vaccine in California. Staffing at the site consisted of 6 persons registering clients for vaccination appointments, 3 persons greeting and checking-in clients, 12 people preparing and administering vaccines, 2 people assisting clients with translation and navigation, 2 people roaming the site (‘community health team’) providing education, answering any client questions, and discussing how to motivate unvaccinated friends and family, 2 persons supporting check-out procedures including vaccine card preparations, and 2 site managers overseeing staff and logistics. The security and safety of clients and staff were extremely important considerations and were provided by Promotores and members of San Francisco’s Community Ambassadors program.
All site personnel were selected based on their desire to serve their community with respect and compassion; they received initial and ongoing training emphasizing the importance of kindness and helpfulness and that the “client” needs should be understood and respected. All site personnel were also provided basic education on key facts related to COVID-19 vaccinations, based on the principle that any community member might ask any staff member basic COVID-19 vaccine questions at any time point in the process, and such question provide important teaching opportunities. Daily morning staff meetings occurred throughout the implementation period, which provided opportunities to discuss ongoing successes and challenges as well as changing COVID-19 vaccine eligibility criteria and any associated necessary adaptations in strategies; they also served as an important opportunity to provide staff refresher trainings on client-centeredness and updated education as new knowledge related to COVID-19 vaccines became available.
Low-barrier registration and vaccination approach. COVID-19 vaccine registration was originally available on-site, but it was quickly moved to the nearby UeS testing site given high demand, long lines and resultant congestion. We initially only provided in-person registration to preferentially provide access to eligible community members for whom online registration, as required at the time for most other vaccination sites, presented a barrier to access. In-person appointment scheduling was available onsite outside of operational hours and served as an opportunity for community members to interact and converse with one another while awaiting scheduling. To remove barriers to registration and vaccine appointment check-in, especially for clients who may have had immigration fears or concerns, community members were able to self-attest to their eligibility. Clients were not required to show any form of identification, or to provide proof of residence, or healthcare insurance in order to be scheduled. Upon arriving at the vaccination site, clients were checked in and scheduled for their second vaccine dose. Following vaccination, clients waited for at least 15 minutes in the post-vaccination waiting area (Fig 2d), where site staff provided them education on adverse side effects to monitor for and ensured that any questions were answered (Table 1). Those clients who did not have insurance or a primary care provider were set up with a community health partner who offered low barrier care in the event that they had any health concerns beyond the standard expected side effects of vaccination. Upon request, clients were provided letters for employers to account for their time away from work. Clients were provided their proof of vaccination only at the end of this observation period.
Leveraging social networks to increase vaccine uptake (“Activate”).
In recognition of the important role that vaccinated individuals play in influencing COVID-19 vaccine knowledge, attitudes and beliefs (and ultimately vaccine uptake) among their friends, family members, and co-workers, we sought to empower clients to become “vaccine ambassadors (Table 1).” During the post-vaccination waiting period, two dedicated staff members (who also provided post-vaccination education and answered any questions), shared their personal experiences, encouraged clients to reach out to members of their social network that had not yet been vaccinated and share their positive vaccination experiences and recommend that they too get vaccinated. This simple act serves multiple functions, including that the peer vaccine ambassador can provide COVID-19 vaccine education and debunk common myths and misconceptions, demonstrate good health-seeking behavior, serve as a credible source for vaccination experiences and also to provide social support to get vaccinated [29–31]. Site staff provided tips on how to handle difficult conversations and role-played different hypothetical scenarios with clients to bolster confidence. All vaccinated clients were provided fliers with UeS neighborhood vaccination site registration information that could be handed out to unvaccinated friends, family members and coworkers. The flyer included a phone number for peer referrals of the vaccinated clients to call with any questions related to COVID-19 vaccines or registration.
Evaluation of the “Motivate, Vaccinate, and Activate” strategy
RE-AIM evaluation measures.
We evaluated the “Motivate, Vaccinate, and Activate” strategy using the Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) framework. We chose the RE-AIM framework as it allowed us to evaluate both individual client-level outcomes (reach and effectiveness), as well as site and community-level outcomes (effectiveness, implementation, and maintenance) [32]. Furthermore, RE-AIM has recently been updated to include an explicit focus on equity and to address dynamic implementation contexts that may require adaptive strategies to maintain interventions over time [24]. This provided an enhanced framework to evaluate our implementation strategy, which sought to facilitate equitable vaccine access and uptake among Latinx community members and also have components that could be adapted to respond to rapidly changing community needs and public health guidance.
- Reach: We sought to reach and increase vaccine uptake among any Latinx adults and adolescents living in San Francisco, with an emphasis on the Mission District, as soon as they became eligible according SFDPH guidance. We conceptualized reach at two levels (proximal and distal). The proximal reach of our vaccination program included the number of individuals directly reached by each implementation strategy component. The distal reach included the number of individuals who were received a COVID-19 vaccination at the UeS neighborhood vaccination site. Because of the community-based design of our implementation strategy, it is difficult to measure proximal reach, e.g., the exact number of Latinx community members who were reached through the community mobilization and demand generation activities, as a result of simply passing by the community-located vaccination site, or through contact with peer vaccine ambassadors. However, because the UeS neighborhood vaccination site was outside of the formal healthcare system, these activities were necessary precursors for community members to become aware of, schedule, and receive a vaccination at the UeS neighborhood vaccination site. Therefore, our evaluation focused on distal reach, e.g., the number of individuals scheduled for vaccination and vaccinated at the UeS neighborhood site. In order to evaluate whether our strategy reached Latinx people (the priority population our strategy was tailored for and aimed to reach) we also evaluated the demographic and socioeconomic characteristics of individuals vaccinated at the neighborhood vaccination site as measures of representativeness. We also assessed measures of geographic coverage by estimating the proportion of all Mission Distract (zip code 94110) residents vaccinated overall and among Latinx individuals as well as the proportion of all vaccinated individuals in San Francisco reached by the UeS neighborhood site overall and among Latinx persons.
- Effectiveness: There is strong evidence, including robust population-level data, that demonstrates that the COVID-19 vaccine, once administered, is highly effective in reducing the risk of COVID-19 disease and transmission [33–36]. Therefore, measures of effectiveness associated with the multicomponent implementation strategy used indicators of behavior change, including the proportion of clients who said that they were able to get vaccinated more quickly had the neighborhood site not existed and the proportion of clients who stated that they were more likely to reach out to and recommend vaccination to their unvaccinated friends, family members and coworkers after their experiences at the UeS neighborhood vaccination site. We also evaluated the proportion of clients at the neighborhood vaccination site who completed their second vaccine dose [37], as this metric may reflect a number of aspects of fidelity to and acceptability of the “Motivate, Vaccinate, and Activate” strategy and is therefore a composite quality outcome measure.
- Implementation: Implementation outcomes assessed were fidelity to each of the implementation strategy components (Motivate, Vaccinate, and Activate) as designed and also the acceptability of the overall implementation strategy among community members vaccinated through the UeS Neighborhood site.
- Maintenance: We evaluated maintenance in two ways. We first assessed temporal trends in the number of individuals receiving their first COVID-19 vaccination—overall and according to both eligibility criteria and ethnicity. This provided insight into the extent to which the “Motivate, Vaccinate, and Activate” strategy was able to evolve over time and mobilize different types individuals as they became eligible, while also being able to consistently reach Latinx individuals throughout the implementation period. We also documented and characterized adaptations during the implementation period.
Data sources and statistics.
Several data sources informed the evaluation of the “Motivate, Vaccinate, and Activate” strategy. Programmatic UeS vaccination data informed reach (including basic demographic characteristics), effectiveness and maintenance related-outcomes. SFDPH surveillance data was used to inform estimates of vaccination coverage [38]. Census data informed population estimates in the Mission District (zip code 94110) [26]. To better understand the characteristics of those being reached, the possible reach and effectiveness of the peer vaccine ambassador strategy component, and the acceptability of our strategy among clients served by the neighborhood vaccination site, we administered a structured survey between May 2, 2021 and May 19, 2021 (S1 Appendix). The survey data also captured additional client information including household income, occupation, insurance status and primary care status. It was administered on-site among those in the waiting area following completion of their vaccination (either first or second dose). Fidelity to and adaptations made to the implementation strategy components were assessed, discussed and documented throughout the implementation period as part of daily meetings with UeS neighborhood vaccination site workers and weekly meetings with UeS leadership.
Data were administratively censored at May 19, 2021, corresponding to a 16-week evaluation period. Analyses were restricted to adolescents and adults 16 years of age and older. Simple descriptive statistics were used to characterize individuals—Fisher’s exact or chi-squared tests were applied, as appropriate. As we primarily aimed to reach Latinx community members through our “Motivate, Vaccinate, and Activate” Strategy, all outcomes were assessed overall and according to whether individuals identified as Latinx (e.g., Latinx versus not-Latinx).
Results
Reach
COVID-19 vaccine uptake.
Overall, there were 12,103 unique individuals registered for a COVID-19 vaccine at the UeS neighborhood vaccination site between February 1 and May 19, 2021, of which 11,098 (91.7%) received at least one vaccine dose at the neighborhood site; the proportion of persons registered who received at least one vaccine dose at the neighborhood site did not differ according to age, sex or ethnicity (S1 Table). In total, 20,792 COVID-19 vaccine doses were administered to community members ≥16 years old during the evaluation period.
The characteristics of 11,098 individuals receiving at least one COVID-19 vaccination at the UeS neighborhood vaccination site are shown in Table 2. Vaccine recipients had a median age of 43 (IQR, 32–56) years, 53.9% were male, and 70.5% were Latinx, 7.7% were Asian, 2.4% were black and 14.1% were white; 50.7% and 14.3% were either a first- or second-generation immigrant, respectively. The majority of clients receiving a vaccine dose worked in front-facing retail jobs and 61.0% of individuals had an annual household income of less than $50,000 per year (Table 2). More than one-third (36.9%) of clients did not have health insurance and nearly half (46.3%) did not have an established primary care provider. Latinx clients were substantially more likely than non-Latinx clients to have an annual household income of less than $50,000 a year (76.1% vs. 33.5%), be a first-generation immigrant (60.2% vs. 30.1%), not have health insurance (47.3% vs. 16.0%), and not have access to primary healthcare services (62.4% vs. 36.2%) (Table 2).
Next, we assessed COVID-19 vaccine coverage associated with the UeS Neighborhood vaccination site. Among all eligible individuals (≥16 years old) estimated to be living in the Mission District (zip code 94110), 5.7% (n = 3,590/62,452) received at least one vaccine dose at the neighborhood site; this included 11.9% (n = 2,484/20,859) of the estimated number of Latinx residents. Compared to the ethnic makeup of the Mission District, clients receiving at least one vaccine dose at the neighborhood vaccination site were far more likely to be Latinx (70.5% vs. 33.4%) and far less likely to be white (14.1% vs. 44.8%) (Fig 3). Notably, vaccinated clients were greater than 5-times more likely to be Latinx than the overall vaccinated population of San Francisco (70.5% vs. 13.3%) (Fig 3). While the neighborhood site was based in the Mission District (zip code 94110), it had broad geographic reach, such that less than one-third (32.3%) of all vaccine recipients resided in the Mission District (Fig 4); the vaccination site was accessed by a large number of predominantly Latinx individuals residing and working throughout San Francisco and the Bay Area (S2 Table). The neighborhood site also appeared to reach persons residing predominantly in the neighborhoods of San Francisco that have been the most impacted by COVID-19 (Fig 4, S1 Fig, S2 Table).
The large majority of vaccinated clients living outside of San Francisco work in the Mission District.
Factors influencing clients to get vaccinated at neighborhood site.
Of 3,597 clients offered to take a survey about their vaccination experiences, 997 (27.7%) completed the survey; compared to those who declined survey participation, survey respondents were slightly younger and less likely to be Latinx (S3 Table). Clients who were vaccinated at the UeS neighborhood vaccination site reported that they had heard about or became aware of the site in a number of different ways (Table 3). Clients most commonly found out about the site was from a friend, family member or co-worker (36.1%); a large number of clients also reported receiving a text invitation on their phone (21.0%), walking past the community-based site (17.8%) and receiving a direct referral from the nearby UeS COVID-19 testing site (11.4%). Clients less commonly cited having been made aware of the site through outreach from a community volunteer, or via a flyer, social media, or news sources (Table 3). The proportion of clients stating that they heard about the site through a friend or family member did not differ substantially by ethnicity (37.8% vs. 32.6%; Table 3). However, compared to non-Latinx clients, Latinx clients were more likely to report hearing about the site by directly passing by it in the neighborhood, and were less likely to have received a direct text invitation (Table 3).
Clients reported that their single most important reason for choosing to get vaccinated at the UeS neighborhood site was (1) because it was in their neighborhood (29.0%), (2) because scheduling was easy and convenient (26.7%) and (3) because someone they trusted had recommended it to them (18.0%) (Table 3). Latinx clients were more likely to choose the site because of its bilingual staff compared non-Latinx clients but were less likely to cite the ease and efficiency of scheduling as an enabling factor (Table 3).
Implementation
Fidelity.
Overall, we were able to deliver each of the components of the “Motivate, Vaccinate, and Activate” strategy as originally intended. As intended by design, the strategy was adapted in response to rapid evolving eligibility criteria and site capacity (Table 4). Most adaptations to the strategy occurred early on and were related to the “Vaccinate” component of the strategy. We aimed to provide timely vaccination to all community members who were eligible and wished to be vaccinated at the neighborhood site. While that was often possible, at times, peaks in demand exceeded our capacity to provide immediate vaccinations. In order to not delay vaccination among highly motivated community members, we partnered and worked closely with a local safety net hospital to extend the reach of our strategy by facilitating referrals for typically either same-day or next-day vaccination appointments. There were two key features of this adaption: (1) free transportation was provided to any referred client who needed it and (2) a UeS community team member went with referred clients or met them at the hospital to provide support and help navigate any additional barriers to getting vaccinated. While complete estimates are not available, more than 2,400 additional community members were directly referred and scheduled for vaccination through this strategy, including approximately 850 during the second week of March 2021. We believe that this was a key adaptation that was needed as to not undermine the overall effectiveness of our strategy, however, it was only required for less than four weeks.
Acceptability.
The UeS neighborhood site was highly acceptable to clients who were vaccinated there. Of 997 clients completing the survey, 98.6% stated that they would recommend the site to others; this did not differ between Latinx and non-Latinx clients (S4 Table). Clients were more likely to say that they would recommend the site to family members (82.1%) and friends (84.5%) than to co-workers (67.1%) (S4 Table).
The features of the neighborhood vaccination site and their vaccination experience that clients stated that they liked the most were (1) the friendly and professional staff (40.3%), and (2) that the process was fast and efficient (32.8%) (S4 Table). Latinx clients were more likely to say that staff friendliness and professionalism was the site feature they liked the most about the neighborhood site, while non-Latinx clients were more likely to state they most liked the overall efficiency of the process (on average wait time was less than 5 minutes from check in) (Fig 5a). Clients stated that they liked and appreciated many additional features of the neighborhood vaccination site and preferences differed by ethnicity (Fig 5b, S4 Table). Latinx clients were more likely to state they liked that the staff was bilingual. Notably, more than a quarter of Latinx (26.3%) and non-Latinx clients (30.5%) reported that they liked that they did not need to show documentation of residency or proof of vaccine eligibility. Very few clients reported any aspects of their experience that they disliked. A few clients noted that they had experienced long wait times but were not bothered by it, while one client felt that the outdoor setting was not private enough. Several people responded that the site experience could be further improved by providing onsite toilets.
Panel A shows the features that clients liked the most (can choose only one) while Panel B shows all of the features that clients stated that they liked.
Maintenance.
Several adaptations were made during the early implementation period, but there were very few subsequent adaptations and the “Motivate, Vaccinate, and Activate” strategy was delivered with high fidelity over time (Table 4). An important feature that allowed us to continue to deliver our strategy with fidelity was that there was consistent staffing that helped facilitate group communication and cohesiveness.
As the eligibility for COVID-19 vaccination shifted over time in San Francisco, the “Motivate, Vaccinate, and Activate” strategy was able to continue to reach and facilitate vaccination of newly eligible community members (Fig 6a). Notably, despite evolution of the eligibility criteria over time, the large majority of clients reached throughout the entire implementation period were Latinx (Fig 6b). While the number of new individuals vaccinated in May 2021 declined (mirroring local and national trends) (Fig 6a), the proportion of daily vaccinations that were among Latinx individuals increased (Fig 6b).
Panel A shows the number of vaccinations that were administered each week according to eligibility criteria indication. Panel B shows the proportion of all vaccinations each week that were administered to Latinx persons; this provides measure of how effectively Latinx individuals were reached throughout the entirety of the implementation period.
Effectiveness.
Indicators of positive behavior change. There were 58.4% of clients that said they got vaccinated sooner than they otherwise would have had the neighborhood vaccination site not existed; this included 56.1% of Latinx clients and 63.2% of non-Latinx clients (Table 5). After their experiences at the neighborhood vaccination site, 90.1% of clients said they were more likely to recommend getting vaccinated to family members, friends, and co-workers; this did not meaningfully differ by ethnicity (Table 5).
Approximately 40% (40.3%) of clients said that they knew at least one person that had yet to be vaccinated; this was slightly higher among Latinx clients than non-Latinx clients (42.9% vs. 34.7%). Of clients who reported knowing unvaccinated individuals, 64.6% reported knowing 3 or more. Among clients who received both vaccine doses (n = 729), 91.0% said that after their first vaccination experience, they personally reached out to at least one unvaccinated person they knew to recommend COVID-19 vaccination; Latinx clients were more likely than non-Latinx clients to reach out and recommend vaccination to 3 or more persons (65.3% vs. 55.9%; Table 5). Notably, 83.0% of clients stated that they were aware of at least 1 family member, friend or co-worker who got vaccinated as a result of their direct outreach; Latinx clients were more likely than non-Latinx persons to report that 3 or more persons got vaccinated a result of their influence (46.7% vs. 36.5%; Table 5).
Next, we evaluated the proportion of clients receiving their first COVID-19 vaccine dose at the UeS neighborhood site, who also completed their second vaccine dose. Among 9,305 clients with at least 4 weeks of follow-up time since their first vaccine dose, 9,152 (98.4%) completed their second dose; the proportion of clients completing both vaccine doses did not differ according to age, sex or ethnicity (S5 Table).
Discussion
From the onset of the COVID-19 pandemic, there have been stark racial and ethnic disparities of populations infected and vaccinated, reflecting known health inequities in the United States [7, 8]. In the setting of a rapidly moving pandemic such as COVID-19, the challenge is how to overcome in a short time period the decades of disparate access to care and resulting mistrust among vulnerable populations in a way that improves health outcomes and can lead to sustained gains in health delivery. Via a community, academic and public health partnership (Unidos en Salud) we developed and evaluated a community-based, “Motivate, Vaccinate, and Activate” strategy. We sought to increase uptake of COVID-19 vaccination among Latinx persons and to activate clients to be community vaccine ambassadors within their social networks.
We effectively reached the target population—70% of more than 11,000 vaccine recipients during the evaluation period were Latinx, the majority of whom were first generation immigrants with a household income of less than $50,000 and without a primary care provider. The geographic reach extended to the Latinx community beyond the surrounding neighborhood to the Southeast sector of San Francisco, the area with the most COVID-19 cases in San Francisco [38]. The vaccination program was highly acceptable, with 99% of clients reporting they would recommend the site to others. The program was also highly effective, as 58% of people reported that they were vaccinated sooner because of the program, and among those who received both vaccine doses, over 90% of clients personally reached out to at least one person in their social network to recommend COVID-19 vaccination. Additionally, 98% of clients completed both vaccine doses, which is higher than early national estimates of 88% [37]. To our knowledge, this is the first evaluation of the implementation and effectiveness of a multicomponent community-based vaccination program designed to reach the Latinx community.
Our data suggest that efforts to address access and trust-related barriers underlie the effectiveness of the program. Access-related barriers drive a large portion of the COVID-19 vaccination disparities between Latinx and non-Latinx White people [14]. Our site was embedded in the San Francisco Mayor’s strategy to offer vaccination through multiple venues, including mass vaccination sites, health care and nursing home settings, pharmacies, and community sites. Local community sites can remove transportation barriers inherent to mass sites, mitigate issues of trust, and reach persons who are not actively connected with a formal health system. Despite all San Franciscan’s being eligible for health care, many vulnerable community members do not identify as having a primary physician or health insurance. Reasons may include the many administrative steps required for health care registration, language barriers, lack of outreach and navigation services on the part of health institutions and mistrust of government institutions based on past negative experiences. Clients at our vaccine site reported geographic convenience outside of a hospital setting and ease of registration (computer access not required) among their top reasons for choosing the neighborhood vaccine site. Time away from work for front-line workers is a major consideration for engagement in health care. The speed and efficiency of the experience was among the top three factors that people appreciated about the site.
Our survey results are consistent with recent national surveys that highlighted the importance of access-related barriers among Latinx persons who had not yet received the vaccine, with concerns about missing work, transport to the site, and information gaps about cost and impact on legal status [14]. Placing vaccine sites in central locations such as such as in Grand Central Station in New York, or trusted community sites such as churches has also yielded promising results [39–42]. While convenience was an important feature of the vaccine site, it alone is likely insufficient, and ensuring trust in the vaccine itself is the first step to getting people to come to the site.
In the “Motivate” component of our model, we used multiple approaches to address concerns voiced by Latinx persons, drawn both from prior work of Unidos en Salud in San Francisco and from national surveys, including safety, cost, eligibility, and effects on immigration status [13, 14]. Strategies to address these concerns included high-touch methods such as ‘door-to-door’ vaccine education and registration and mobilization by trusted community leaders via their social networks. We also employed less resource-intensive vaccine promotion strategies such as Spanish-language media, in which Unidos En Salud community leaders provided information and answered questions on COVID-19 vaccines. Our community team posted flyers in the neighborhood, including at local businesses, and handed out educational information about vaccines at our adjacent COVID-19 testing site.
Our data highlight the importance of trusted messengers in the decision to come to the community vaccine site [31]. Nearly 20% of clients said that the most important factor in their decision to come to the vaccine site was because someone they trusted recommended it to them, as opposed to less than 5% who heard about the site through a flyer in the community or media campaigns. These data are consistent with prior data on the positive impact of door to-door outreach [43] and endorsements from trusted community members on increasing vaccination from influenza [44], childhood vaccines [45], and HPV vaccines [46]. Our findings are also consistent with a multicomponent intervention involving mobile clinics and religious leaders as vaccine ambassadors that led to high uptake among COVID-19 vaccines among Black people living in a community in Southern California [42]. Multi-pronged approaches to community-led education and outreach can increase trust in vaccine safety, effectiveness, and the healthcare system, and are fundamental to facilitating forward movement along the entire continuum of vaccine hesitancy [44–49].
Paramount to our strategy was to create a convenient, language-concordant, and welcoming vaccination site. The client experience at the site, including efficiency, and access to bilingual staff and health education in the post-vaccination area likely amplified trust in both the vaccination site and the vaccine itself. This was evidenced by the finding that 99% of vaccinated clients reported that they would recommend the site to their friends or family members, and that nearly two-thirds recommended COVID-19 vaccination to 3 or more people in their social network. Additionally, friendly and professional staff were the features Latinx clients liked most about the neighborhood vaccination site.
Peer-referrals and social network interventions can increase trust in marginalized communities and rapidly diffuse innovations. It was notable that even when San Francisco exceeded more than 75% coverage of COVID-19 vaccination among residents ≥16 years old, the social networks of people at our vaccine site still included large numbers of unvaccinated friends and family members. That our clients most frequently heard about the UeS neighborhood vaccination site from a friend or family member (36%) and more than 80% of vaccinated clients then positively influenced an unvaccinated person they knew to get vaccinated strongly supports the value of social network interventions in our setting and their potential to build trust and reach unvaccinated people. Our community health team, a bilingual bicultural team of community members, provided additional education about COVID-19 vaccines, testing, answered related questions and shared their own experiences about encouraging friends and loved ones to get vaccinated. We hypothesize that these positive interactions addressed information gaps and empowered people to become ‘vaccine ambassadors,’ and reach out to their unvaccinated family members, friends and co-workers. These findings build upon a growing body of literature demonstrating the effectiveness of social network interventions for positive health promotion, including prevention strategies for HIV (PrEP), STI and HIV testing, as well as other health behaviors and outcomes including smoking cessation, alcohol misuse, and improving hemoglobin A1c levels in persons with diabetes [17].
Although most factors influencing vaccination at our site were similar across race and ethnicity, Latinx compared to non-Latinx were more likely to report that bilingual staff were an important factor for choosing the site. This highlights the importance of language and cultural concordance throughout all stages from community outreach and mobilization, through vaccination to address structural barriers, information-gaps about vaccine eligibility, and perceptions that the COVID-19 vaccine costs money and can impact one’s immigration status. Additionally, requirements to provide documentation increases access-related barriers, especially for first generation immigrants and people who work informal jobs. Over 40% of Latinx persons in a national survey cited concern about having to provide documentation and 40% feared that the process would impact their legal status [14]. To address these concerns and to lower the barrier to vaccinations, we did not require identification or proof of vaccine eligibility at our vaccine site, and approximately one third of clients reported that they appreciated this feature. Removing the requirement for identification or proof of residence or employment should be considered in the design of low-barrier vaccine sites.
Our implementation strategy quickly adapted to changes in vaccine eligibility which created surges in demand over time. (Table 4). Initially vaccine demand exceeded supply. To address this need and not turn people away, we expanded our vaccine site to also include a vaccine navigation hub and become a gateway to a higher volume vaccine site at the safety-net hospital nearby. To facilitate access to larger vaccine sites, we arranged free transportation, helped schedule appointments on-site, and had our community workers accompany clients to the hospital site in order to overcome mistrust and fear of formal health care systems. Later, as supply exceeded demand, we shifted our mobilization strategy away from posting flyers in the community and harnessing Spanish language media, to more individualized ‘one-on-one’ discussions and also a focus on a social network-based approach as persons at our vaccine still had a large number of people in their network who remained unvaccinated. Financial and non-financial incentives can be effective in promoting vaccinations and adaptations that include incentives are worth further consideration and study [45, 50, 51].
The evaluation of our program has some limitations. Our methods underestimate the program’s reach, as we could not quantify the number of people who were influenced by our multifaceted, community-based demand generation activities, but who were vaccinated at a different site. Additionally, our reported reach does not include over 2,000 direct referrals to the vaccination site at the nearby county hospital. Secondly, the structured survey on clients’ experiences was only completed during the period of general eligibility, and experiences may have differed compared to the beginning of the program. This can also be seen as a strength, as the findings are more generalizable to the current vaccine landscape—where supply is greater than demand and all adults are eligible for the vaccine. As with most multi-component interventions, we are unable to fully disentangle the relative effects of the different components and subcomponents of the overall strategy. There are also some limitations to our measurements of effectiveness; though a high proportion of people reported that someone they referred received a vaccine, we could not measure this directly. However, even if the peer referral did not result in a vaccination, it is likely that the referral served as a nudge further down the continuum towards vaccine confidence.
In conclusion, our “Motivate, Vaccinate, and Activate” vaccine promotion strategy reached a high proportion of Latinx residents in San Francisco. We attribute the success of the program to demand generation through trusted messengers and social networks, multi-faceted and adaptable mobilization strategies, and a convenient and welcoming neighborhood vaccine site. Our Unidos en Salud community, academic, and public health partnership and co-design was fundamental to the program and cannot be underestimated. Though this program was geared towards addressing the specific barriers and needs of the Latinx community in San Francisco, the fundamental pillars of this program can be adapted to other local contexts.
Supporting information
S1 Fig. Map of San Francisco according to cumulative COVID-19 cases (rate per 10,000 residents) according to neighborhood.
Darker blue shading indicates a higher cumulative prevalence of COVID-19 in a given neighborhood. The pink dot indicates the location of the Unidos en Salud neighborhood vaccination site.
https://doi.org/10.1371/journal.pone.0257111.s001
(TIF)
S1 Table. Characteristics of individuals who were registered for vaccination at the Unidos en Salud vaccination site between February 1 and May 19, 2021, according to whether they did or did not receive at least one vaccine dose at the Unidos en Salud vaccination site.
https://doi.org/10.1371/journal.pone.0257111.s002
(DOCX)
S2 Table. Geographic residence of clients receiving at least one vaccine dose at the Unidos en Salud neighborhood vaccination site between February 1 and May 19, 2021 according to zip code.
https://doi.org/10.1371/journal.pone.0257111.s003
(DOCX)
S3 Table. Characteristics of individuals who were offered participation in the on-site post-vaccination survey, according to whether they did or did not agree to complete the survey.
https://doi.org/10.1371/journal.pone.0257111.s004
(DOCX)
S4 Table. Acceptability measures associated with the Unidos en Salud vaccination site.
https://doi.org/10.1371/journal.pone.0257111.s005
(DOCX)
S5 Table. Characteristics of clients receiving at least one vaccine dose at the Unidos en Salud vaccination site between February 1 and April 21, 2021, according to whether they completed both vaccine doses.
https://doi.org/10.1371/journal.pone.0257111.s006
(DOCX)
Acknowledgments
We would like to thank the community members who participated in this initiative and the many vaccination site staff, community ambassadors, and volunteers. We thank the Chan Zuckerberg Initiative, Supervisor Hillary Ronen, Mayor London Breed, The San Francisco Department of Public Health and Dr. Grant Colfax, Dr. Naveena Bobba, Dr. Susan Phillips, Dr. Mary Mercer, Emily Reingold, and Dr. Ellen Chen. We gratefully acknowledge the San Francisco Latino Task Force-Response to COVID-19, Bay Area Phlebotomy and Laboratory Services (BayPLS), Stacy Powers and BRAVA for Women in the Arts, Bevan Dufty and the BART team, PrimaryBio COVID-19 Testing Platform, Dr. Rachel Stern, Phillipa Doyle, and Ricardo Duarte from Zuckerberg San Francisco General Hospital, Dr. Jessica Briggs and Dr. Susa Coffey from UCSF, and Dara Fonseca, Jocelin Payan, and Jack Fukushima from Unidos en Salud. We also would like to thank Mike Kai Chen for the use of his photographs in this manuscript. We also gratefully acknowledge the OpenStreetMap contributors, who provided the data base layer used in Fig 4 and S1 Fig. OpenStreetMap data is available under the Open Database License. For more information, see www.openstreetmap.org/copyright.
References
- 1. Chamie G, Marquez C, Crawford E, Peng J, Petersen M, Schwab D, et al. SARS-CoV-2 Community Transmission disproportionately affects Latinx population during Shelter-in-Place in San Francisco. Clin Infect Dis. 2020;73: ciaa1234-. pmid:32821935
- 2. Gil RM, Marcelin JR, Zuniga-Blanco B, Marquez C, Mathew T, Piggott DA. COVID-19 Pandemic: Disparate Health Impact on the Hispanic/Latinx Population in the United States. J Infect Dis. 2020; jiaa474-. pmid:32729903
- 3. Kullar R, Marcelin JR, Swartz TH, Piggott DA, Gil RM, Mathew TA, et al. Racial Disparity of Coronavirus Disease 2019 (COVID-19) in African American Communities. J Infect Dis. 2020;222: jiaa372-. pmid:32599614
- 4. Rodriguez-Diaz CE, Guilamo-Ramos V, Mena L, Hall E, Honermann B, Crowley JS, et al. Risk for COVID-19 infection and death among Latinos in the United States: examining heterogeneity in transmission dynamics. Ann Epidemiol. 2020;52: 46–53.e2. pmid:32711053
- 5. Bibbins-Domingo K, Petersen M, Havlir D. Taking Vaccine to Where the Virus Is—Equity and Effectiveness in Coronavirus Vaccinations. Jama Heal Forum. 2021;2: e210213.
- 6. Hooper MW, Nápoles AM, Pérez-Stable EJ. No Populations Left Behind: Vaccine Hesitancy and Equitable Diffusion of Effective COVID-19 Vaccines. J Gen Intern Med. 2021; 1–4. pmid:33754319
- 7.
Ndugga N, Pham O, Hill L, Artiga S, Alam R, Parker N. Latest Data on COVID-19 Vaccinations Race/Ethnicity. https://www.kff.org/coronavirus-covid-19/issue-brief/latest-data-on-covid-19-vaccinations-race-ethnicity/. Accessed May 20, 2021.
- 8.
California Department of Public Health. Vaccination progress data—Coronavirus COVID-19 Response. https://covid19.ca.gov/vaccination-progress-data/. Accessed May 20, 2021.
- 9. Nuriddin A, Mooney G, White AIR. Reckoning with histories of medical racism and violence in the USA. Lancet. 2020;396: 949–951. pmid:33010829
- 10. Martinez O, Wu E, Sandfort T, Dodge B, Carballo-Dieguez A, Pinto R, et al. Evaluating the Impact of Immigration Policies on Health Status Among Undocumented Immigrants: A Systematic Review. J Immigr Minor Healt. 2015;17: 947–970. pmid:24375382
- 11. Saadi A, Molina US, Franco-Vasquez A, Inkelas M, Ryan GW. Assessment of Perspectives on Health Care System Efforts to Mitigate Perceived Risks Among Immigrants in the United States. Jama Netw Open. 2020;3: e203028. pmid:32301990
- 12. Morey BN. Mechanisms by Which Anti-Immigrant Stigma Exacerbates Racial/Ethnic Health Disparities. Am J Public Health. 2018;108: 460–463. pmid:29470116
- 13. Peng J, Marquez C, Rubio L, Chamie G, Jones D, Jacobo J, et al. High likelihood of accepting COVID-19 vaccine in a Latinx community at high SARS-CoV2 risk in San Francisco. Open Forum Infect Dis. 2021; ofab202-.
- 14.
Kaiser Family Foundation. KFF Covid-19 vaccine monitor: April 2021. https://www.kff.org/coronavirus-covid-19/poll-finding/kff-covid-19-vaccine-monitor-april-2021/. Accessed May 21, 2021.
- 15. Williams DR, Cooper LA. Reducing Racial Inequities in Health: Using What We Already Know to Take Action. Int J Environ Res Pu. 2019;16: 606. pmid:30791452
- 16. Fisk RJ. Barriers to Vaccination for COVID-19 Control—Experience from the United States. Global Heal J. 2021;5: 51–55. pmid:33585053
- 17. Hunter RF, de la Haye K, Murray JM, Badham J, Valente TW, Clarke M, et al. Social network interventions for health behaviours and outcomes: A systematic review and meta-analysis. Plos Med. 2019;16: e1002890. pmid:31479454
- 18. Marcelin JR, Swartz TH, Bernice F, Berthaud V, Christian R, da Costa C, et al. Addressing and Inspiring Vaccine Confidence in Black, Indigenous, and People of Color (BIPOC) during the COVID-19 Pandemic. Open Forum Infect Dis. 2021.
- 19. Drahota A, Meza RD, Brikho B, Naaf M, Estabillo JA, Gomez ED, et al. Community-Academic Partnerships: A Systematic Review of the State of the Literature and Recommendations for Future Research. Milbank Q. 2016;94: 163–214. pmid:26994713
- 20. Fields J, Gutierrez RJ, Marquez C, Rhoads K, Kushel M, Fernandez A, et al. Community-Academic Partnerships to Address COVID-19 Inequities: Lessons from the San Francisco Bay Area. NEJM Catal. 2021.
- 21. Kerkhoff AD, Sachdev D, Mizany S, Rojas S, Gandhi M, Peng J, et al. Evaluation of a novel community-based COVID-19 ‘Test-to-Care’ model for low-income populations. Plos One. 2020;15: e0239400. pmid:33035216
- 22. Rubio LA, Peng J, Rojas S, Rojas S, Crawford E, Black D, et al. The COVID-19 Symptom to Isolation Cascade in a Latinx Community: A Call to Action. Open Forum Infect Dis. 2021;8: ofab023. pmid:33623805
- 23. Pilarowski G, Lebel P, Sunshine S, Liu J, Crawford E, Marquez C, et al. Performance characteristics of a rapid SARS-CoV-2 antigen detection assay at a public plaza testing site in San Francisco. J Infect Dis. 2021;223: jiaa802-. pmid:33394052
- 24. Shelton RC, Chambers DA, Glasgow RE. An Extension of RE-AIM to Enhance Sustainability: Addressing Dynamic Context and Promoting Health Equity Over Time. Frontiers Public Heal. 2020;8: 134. pmid:32478025
- 25.
Urban Displacement Project. Rising Housing Costs and Re-Segregation in San Francisco. 2018 Sep. https://www.urbandisplacement.org/sites/default/files/images/sf_final.pdf
- 26.
Census Reporter. ZIP Code Tabulation Area 94110, San Francisco, CA. https://censusreporter.org/profiles/86000US94110-94110/. Accessed May 21, 2021.
- 27.
The San Francisco Latino Task Force on COVID-19. 2 Jul 2020 [cited 2 Jul 2020]. https://www.ltfrespuestalatina.com/
- 28. Porter CM. Revisiting Precede–Proceed: A leading model for ecological and ethical health promotion. Health Educ J. 2016;75: 753–764.
- 29. Funnell MM. Peer-based behavioural strategies to improve chronic disease self-management and clinical outcomes: evidence, logistics, evaluation considerations and needs for future research. Fam Pract. 2010;27: i17–i22. pmid:19509083
- 30. Fisher EB, Ballesteros J, Bhushan N, Coufal MM, Kowitt SD, McDonough AM, et al. Key Features f Peer Support In Chronic Disease Prevention And Management. Health Affair. 2017;34: 1523–1530. pmid:26355054
- 31.
Medicine NA of S Engineering, and. Strategies for Building Confidence in the COVID-19 Vaccines. 2021.
- 32. Glasgow RE, Harden SM, Gaglio B, Rabin B, Smith ML, Porter GC, et al. RE-AIM Planning and Evaluation Framework: Adapting to New Science and Practice With a 20-Year Review. Frontiers Public Heal. 2019;7: 64. pmid:30984733
- 33. Polack FP, Thomas SJ, Kitchin N, Absalon J, Gurtman A, Lockhart S, et al. Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine. New Engl J Med. 2020;383: 2603–2615. pmid:33301246
- 34. Baden LR, Sahly HME, Essink B, Kotloff K, Frey S, Novak R, et al. Efficacy and Safety of the mRNA-1273 SARS-CoV-2 Vaccine. New Engl J Med. 2020;384: 403–416. pmid:33378609
- 35. Haas EJ, Angulo FJ, McLaughlin JM, Anis E, Singer SR, Khan F, et al. Impact and effectiveness of mRNA BNT162b2 vaccine against SARS-CoV-2 infections and COVID-19 cases, hospitalisations, and deaths following a nationwide vaccination campaign in Israel: an observational study using national surveillance data. Lancet. 2021;397: 1819–1829. pmid:33964222
- 36. Benenson S, Oster Y, Cohen MJ, Nir-Paz R. BNT162b2 mRNA Covid-19 Vaccine Effectiveness among Health Care Workers. New Engl J Med. 2021;384: 1775–1777. pmid:33755373
- 37. Kriss JL, Reynolds LE, Wang A, Stokley S, Cole MM, Harris LQ, et al. COVID-19 Vaccine Second-Dose Completion and Interval Between First and Second Doses Among Vaccinated Persons—United States, December 14, 2020−February 14, 2021. Morbidity Mortal Wkly Rep. 2021;70: 389–395. pmid:33735162
- 38.
San Francisco Department of Public Health: COVID-19 Data and Reports. https://data.sfgov.org/stories/s/fjki-2fab. Accessed May 23, 2021.
- 39.
Cabanatuan M. Line around the block for COVID vaccinations in East Oakland. San Francisco Chronicle. 4 Apr 2021. https://www.sfchronicle.com/local/article/Line-around-the-block-for-COVID-shots-in-East-16075137.php
- 40.
Zaveri M. New York will offer vaccines at some subway stops through a pilot program. New York TImes. 10 May 2021. https://www.nytimes.com/2021/05/10/nyregion/nyc-vaccine-subway.html
- 41.
Weiland N. In New Vaccination Push, Biden Leans on His ‘Community Corps.’ New York Times. 16 May 2021. https://www.nytimes.com/2021/05/16/us/politics/coronavirus-vaccination.html
- 42. Abdul-Mutakabbir JC, Casey S, Jews V, King A, Simmons K, Hogue MD, et al. A three-tiered approach to address barriers to COVID-19 vaccine delivery in the Black community. Lancet Global Heal. 2021;9: e749–e750. pmid:33713634
- 43. Prevention C for DC and. Vaccine-preventable diseases: improving vaccination coverage in children, adolescents, and adults. A report on recommendations from the Task Force on Community Preventive Services. MMWR Morb Mortal Wkly Rep. RR-8: 1–15.
- 44. Peterson P, McNabb P, Maddali SR, Heath J, Santibañez S. Engaging Communities to Reach Immigrant and Minority Populations: The Minnesota Immunization Networking Initiative (MINI), 2006–2017. Public Health Rep. 2019;134: 241–248. pmid:30912998
- 45. Banerjee AV, Duflo E, Glennerster R, Kothari D. Improving immunisation coverage in rural India: clustered randomised controlled evaluation of immunisation campaigns with and without incentives. Bmj. 2010;340: c2220. pmid:20478960
- 46. Gottvall M, Tydén T, Höglund AT, Larsson M. Knowledge of human papillomavirus among high school students can be increased by an educational intervention. Int J Std Aids. 2010;21: 558–562. pmid:20975088
- 47. Jarrett C, Wilson R, O’Leary M, Eckersberger E, Larson HJ, Hesitancy the SWG on V. Strategies for addressing vaccine hesitancy—A systematic review. Vaccine. 2015;33: 4180–4190. pmid:25896377
- 48. MacDonald NE, Hesitancy the SWG on V. Vaccine hesitancy: Definition, scope and determinants. Vaccine. 2015;33: 4161–4164. pmid:25896383
- 49. Force TAHCT, Spleen AM, Kluhsman BC, Clark AD, Dignan MB, Lengerich EJ. An Increase in HPV-Related Knowledge and Vaccination Intent Among Parental and Non-parental Caregivers of Adolescent Girls, Age 9–17 Years, in Appalachian Pennsylvania. J Cancer Educ. 2012;27: 312–319. pmid:22131065
- 50.
Vavreck L. $100 as Incentive to Get a Shot? Experiment Suggests It Can Pay Off. New York Times. 4 Mar 2021. https://www.nytimes.com/2021/05/04/upshot/vaccine-incentive-experiment.html
- 51.
McNamara D. Winning Idea: Ohio Vaccine Lottery Shows Some Incentives May Work. Medscape Medical News. 18 May 2021.