Intervention components

The initial idea for this intervention was developed based on earlier work on care seeking [17] and health information-seeking behavior [19], in which we identified reasons for delays in care, and what people needed to timely seek care. For detailed background information and the rationale behind the intervention components, please refer to the original publications [17,19], S1 File (a brief description of the intervention development) and S2 File (the logic model). In brief, these studies describe that women and men from ethnic minority groups preferred receiving information about IHD via culturally-sensitive, live, interactive information sessions organized in community settings. This toolkit was also designed to be gender-informed, by including information on sex differences in symptoms and risk factors, discussing gendered barriers to care, and gender-responsive by taking into account sex-specific preferences for information sessions, e.g., separating women and men. We hypothesize that knowledge on IHD symptoms and when to seek care will improve IHD symptom recognition, and thereby, improve care seeking for potential IHD symptoms. We use the RE-AIM framework to study the toolkit’s reach, effectiveness, adoption, implementation, and maintenance in a real-world setting. Table 1 provides an overview of the tools in this toolkit, which were used to organize the live information sessions. We subsequently evaluated the Reach, Effectiveness, and Adoption of the live information sessions, and the Implementation and Maintenance of the toolkit, described in more detail below.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Tools provided in the toolkit.

https://doi.org/10.1371/journal.pone.0344093.t001

Implementation strategies and settings

During the pilot-implementation study, which took place in May of 2024, the goal was to implement the toolkit by organizing information sessions in four different community settings: a primarily South-Asian Surinamese Mosque, a primarily West-African church, a Turkish mosque, and a physical location rented by South-Asian Surinamese foundation. We aimed to implement the intervention in several different settings representing large ethnic groups in the Netherlands, as this diversity strengthens the external validity. Due to financial and time constraints, we were restricted to four settings at the time of the pilot-implementation, yet, we believe these provide us with a diverse experiences for the pilot study. These specific settings were chosen because these communities represent some of the largest ethnic groups in the Netherlands, and based on willingness of the community leader to co-organize the information sessions.

Based on earlier work [19], we co-organized the information sessions with people from the communities. This required establishing good rapport with community leaders, and discussions around the facilities needed to organize the session, and how we could ensure the session aligned with attendees’ cultural values. For instance, depending on community’s preferences, the information session had a more festive (West-African church and South-Asian Surinamese foundation) or more informative nature (both mosques). Moreover, together with the South-Asian Surinamese foundation, we co-created a play to address IHD symptoms and barriers to care, and simultaneously reduce the taboo on the topic of IHD. A few weeks before the information session, we trained the presenters, and provided them with slides, lessons plans, and additional information. Presenters were able to tailor the slides to community’s preferences and needs.

The information sessions consisted of two parts. First, there was a 45 min interactive presentation on the heart, (sex differences in) IHD symptoms (S3 File) and risk factors, and when to seek medical support. Slides were presented by a healthcare professional from a similar ethnic/cultural background, in the community’s native language. Second, with support from the Dutch Heart Foundation, we organized ‘checkpoints’ during these information sessions, as earlier work showed this would be a good motivator for people to attend an information session [19]. These checkpoints included free blood pressure-, blood glucose- and cholesterol- (both through finger prick), Body Mass Index-, and waist circumference measurements, along with personalized health advice. After the presentation and checkpoints, we shared informational flyers about IHD symptoms which were designed and tested with the target audience (S3 File).

Evaluation methods and data collection

Lastly, the pilot-implementation of the toolkit was evaluated using a mixed-methods design, based on the RE-AIM framework (Table 2). Our quantitative evaluation included anonymous questionnaires (available in B1-level Dutch, English, or Turkish, S4 File) distributed amongst attendees of the information session, and data from the checkpoints, which were written down by trained volunteers performing the measurements. Our qualitative methods included observation forms (S5 File) filled out by 2–3 researchers during the information sessions, focus groups with attendees, and reflections with presenters, community leaders (influential figures within the community, e.g., the Imam and Pastor), and potential maintenance organizations (i.e., organizations in the social- and care domain that potentially have the structure and funding to maintain and scale the intervention). For readability, the evaluation strategies will be structured according to the RE-AIM framework.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. RE-AIM framework applied to the evaluation of a gender-informed and culturally-sensitive toolkit to promote symptom recognition and adequate care seeking in women and men across ethnic groups.

https://doi.org/10.1371/journal.pone.0344093.t002

Data analysis

With respect to the questionnaire data, knowledge improvement was determined by calculating the change in the proportion of respondents who correctly identified (sex differences in) IHD symptoms and risk factors before and after the presentation. Checkpoint data were analyzed by the Dutch Heart Foundation by calculating the number and proportions of elevated values. Data from observation forms and reflections with community leaders, presenters, and potential maintenance organizations were summarized and compared across settings. Finally, all focus group audio recordings were transcribed verbatim, anonymized, and analyzed using inductive coding and a thematic analysis by [author1], [author4], and [author2]. During data analysis, regular sessions were held amongst coders to discuss codes and emerging themes.

Ethical considerations

The study was approved by the [institution] Medical Ethical Review Committee (reference W21_544 #21.600). Due to ethical and privacy regulation, we could not include people younger than 18 years old. Prior to enrollment, focus group participants provided written informed consent and oral approval to participate. Audio recordings were uploaded to the secured servers at [institution], in a folder only accessible by the lead researchers. Audio files and transcripts were transferred via a secured file transfer service. In line with [institution] privacy regulations, signed informed consent forms and paper surveys were saved in a secured cabinet at [institution]. Focus groups were anonymized during transcription, after which audio files were deleted. Interns who supported data collection signed a nondisclosure agreement. To limit the burden for participants with limited levels of literacy, information about the aim of the questionnaire and voluntary nature of participation was shared verbally prior to handing out the questionnaire. The questionnaire itself was anonymous, and participation in the session was not registered by name. Therefore, we did not collect formal informed consent forms for questionnaire participants. Ethical approval for the checkpoints was arranged by the Dutch Heart Foundation.

Reach

The number of attendees varied across data sources and settings. In total, there were 139 completed questionnaires (varying between seven to 46 across settings), 240 attendees according to observation forms (varying between 20–100 across settings) and 193 in checkpoint data (varying between 29–64 across settings). All data sources show that more women than men attended the information sessions (Tables 3 and 4). According to questionnaire data, 98 individuals (71%) who completed the questionnaire fell within the age range of 30–70, i.e., our target audience. Moreover, across data sources, the average age varied across settings, as the lowest average age (mean [standard deviation]) age found in the primarily West-African church (41.4 years [9.8]), versus the oldest age in the primarily South-Asian Surinamese mosque (68.0 years [7.5]). Among those measured at the checkpoints, we found relatively high numbers of elevated values, varying from 23% with high blood glucose to 70% with a large waist circumference (Table 5). Most attendees were reached by invitations from community leaders, advertisements on social media, and word-of-mouth advertising, and reasons for attending the information session included: 1) interest in/a need for information on the topic, 2) free health checks, and 3) habit, i.e., they frequently attend community gatherings.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Questionnaire data on sociodemographic characteristics, attendees’ perceived ability to identify a heart attack, and attendees’ satisfaction.

https://doi.org/10.1371/journal.pone.0344093.t003

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. The number and characteristics of people measured at the checkpoints.

https://doi.org/10.1371/journal.pone.0344093.t004

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Results from checkpoint measurements.

https://doi.org/10.1371/journal.pone.0344093.t005

‘They announced it […]. The flyers were printed and we had to share them with the community. So yeah, I was in church that Sunday and the flyers were given to us, so I had to share it.’

- Woman from primarily West-African Church

Effectiveness

We found small short-term improvements in knowledge on (sex differences in) IHD symptoms and risk factors (Table 6). For instance, different IHD symptoms were correctly identified by 53.4–74.4% of participants in the pre-test, compared to 67.6–85.4% in the post-test. Similarly, numbers increased from 15.1–81.5% to 42.7–88.5% for different cardiovascular risk factors, and 30.5–60.2% to 52.2–75.0% for sex differences in IHD symptoms.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Questionnaire data on respondents’ ability to identifying IHD symptoms and risk factors.

https://doi.org/10.1371/journal.pone.0344093.t006

We organized five focus groups across three settings, with a total of 32 participants (19 women and 13 men). Overall, participants said that they found the session informative, and that they learned new information about IHD symptoms, risk factors, and in particular sex differences in IHD symptoms. For instance, when asked if participants were previously aware of sex differences in IHD symptoms and risk factors, one woman stated:

“For me, no. First time. And I was surprised as well because I didn’t think menopause would have anything to do with a heart attack.”

- Woman from primarily West-African Church

Additionally, most participants felt confident in their ability to recognize a heart attack after the session. Some participants stated they would have liked to receive more in-depth information, however, this mostly concerned people diagnosed with IHD, who were not the target audience. Focus group participants also said the effectiveness could be improved by increasing the length of the presentation, making it more visual, and through repetition of the message both during the presentation and over time.

‘It’s a one-off, yes, all information is given at once and you will only remember certain things. […] Repetition would be better for us. Also because most of the seniors are 65 + . They don’t remember things as quickly and so that’s why repetition is important. ‘

- Man from predominantly South-Asian Surinamese mosque

Adoption

We planned to organize four information sessions in our pilot-implementation study. In total, we contacted five communities with the request to co-organize an information session; one community, however, was unable to host a session due to other priorities. Nevertheless, this shows a high level of adoption amongst the contacted communities, which is further exemplified by additional requests for information sessions in other communities after the pilot-implementation study.

Reflections with community leaders’ revealed that their main motivation for collaborating and hosting such an information session was the high level of interest in heart disease. They often recognized IHD as a large issue in their community, and they felt that their organization had a responsibility in sharing information. Other motivations for hosting an information session included the co-creative approach taken in the process, and the pleasant collaboration between community leaders and researchers. In some communities, it was necessary to visit the community several times to develop trust with community leaders and members, in particular communities that experienced more mistrust towards Dutch institutions. Across settings, an important facilitator for implementation was the prior experience in organizing community events by the community. Challenges to the adoption mostly related to a lack of clear communication concerning finances and planning of the information sessions, as discussing expectations and setting appointments was deemed important to prevent miscommunication.

Implementation

The implementation of the intervention varied somewhat across settings, given that the intervention components were tailored to community preferences, including, e.g., the degree of festivity of the information session. Beyond the tailored intervention components, we observed several instances of infidelity to the implementation. For instance, while the ability to adjust slides to the audience was considered a strength, one presenter took out the interactive components, while interactive and narrative information was used to tailor the information to different cultural backgrounds. Focus group participants in this community said that the lack of interaction led to lower satisfaction, and participants from other communities stated that interactive components such as the anecdotal patient story, were very important and impressive:

“[The anecdotal patient story], was a good example […]. First, she has a daughter, and then comes a divorce, and after that comes the grieve, and before she even grieved, she has a heart attack”

- Woman from Turkish mosque

Moreover, during the final rehearsal of the play, it appeared that the scriptwriters from the community had altered the description of IHD symptoms, which was no longer medically accurate. One final point concerning fidelity was related to the presenter, as one of the presenters was not a medical doctor. This presenter did not possess enough medical knowledge to answer all the questions from the audience, which led to lower satisfaction.

‘He’s not a doctor, […] and he has very little knowledge about the heart. Maybe he prepared well in advance, but not enough to be able to answer all questions.’

- Man from Turkish mosque

Other points of improvement during the implementation were related to the organization of the session, including organizational and technical difficulties that hindered the information sessions. This included disruptions during the presentation when food, drinks, or health checks were offered during the presentation, and issues with the internet connection, microphones malfunctioning, and the visibility of the screen for those seated in the back. Finally, while women and men were seated separately in both mosques, women in one mosque said they still felt uncomfortable sharing personal questions regarding, e.g., menopause in the presence of men.

‘Yes, the doctor asked about menopause, but you can’t say that right in front of the men. I wanted to respond, but the men are there, so you can’t say that publicly. So if it was just a group of women, we could tell you more.’

- Woman from Turkish mosque

Maintenance

Challenges and opportunities for maintenance of the toolkit were discussed with several potential maintenance organizations. Two main prerequisites for maintenance of the toolkit and continuation of the intervention emerged: 1) experience with, and existing infrastructure for, organizing community events, and 2) one individual responsible for the implementation and maintenance. This person in charge requires sufficient training in using the toolkit to promote fidelity in the adoption of the toolkit. While all organizations expressed high levels of interest in the intervention, the willingness to use the intervention was lower in organizations that had no experience in organizing community events. Other opportunities for maintenance included offering the toolkit as an addition to pre-existing activities within organizations, as well as promoting the toolkit amongst communities in order for them to organize information sessions self-sufficiently.

The maintenance of the intervention was discussed with several parties during the study. The intervention has since been hosted by Municipal Health Services Haaglanden. This organization works with community health ambassadors, who have a large network within different ethnic communities, and organize various different activities in the area of health promotion. These health ambassadors were instructed how to use the toolkit and share the information within different communities, including which information could be adapted to fit communities’ preferences and needs, and which information encompassed core elements and should not be changed.

Limitations

Several limitations to this work should be considered. One main limitation pertains to the data collection methods, including the questionnaire and focus group discussions. First, there were large discrepancies in the number of attendees, people measured, and completed questionnaires. This can partially be explained by the presence of minors in some settings (specifically the West-African church), but may particularly reflect a low, potentially selective willingness to fill in the questionnaire. Beyond the small sample size, a large number of participants did not complete the post-test questionnaire, resulting in large numbers of excluded participants and lower reliability of the questionnaire data. Therefore, while questionnaire results are consistent with focus group findings on small, short-term knowledge improvements, the questionnaire data should be interpreted cautiously, especially given that questionnaire drop-out may be selective, and those who dropped out at the post-test may not have participated actively during the presentation. This further suggests that for future evaluations, other evaluation strategies may be considered, including more interactive strategies that do not require literacy, such as actor depicting symptoms and verbally discussing what people the right action would be. Second, social desirability bias could have affected people’s answers during focus group discussions [22]. To mitigate this, we explicitly asked participants to answer honestly, and applied data triangulation by interviewing a diverse range of participants and professionals, which provided a broad range of perspectives on the topic. Finally, evaluation results concerning, e.g., increased knowledge on IHD, are limited to the short-term, and do not provide insights into long-term effectiveness. The long-term effectiveness of the intervention is an interesting direction for future research, also given the lack of research on the long-term effects of one-time interventions on knowledge improvement.

Reach

Despite some variations across settings, the interventions achieved a relatively large reach. Women and somewhat older individuals appeared to be slightly overrepresented, which may reflect the typical composition of these community organizations, or may signal a greater interest in heart disease among these groups. The presence of individuals with IHD was largely unknown, with the exception of some focus group participants who disclosed their condition. However, the intervention appeared to successfully reach the target audience based on age and risk factors. Previous research suggests [23], that defining a target audience should consider not only the number of people and the proportion of at-risk individuals, but also practical aspects such as resources and the persuasibility and accessibility of the population, as well as ethical considerations such as equity. Thus, to further specify the target audience, it is important to take into account that IHD risk may present earlier in certain high-risk ethnic groups [24,25], and therefore, certain populations may be targeted at a younger age. Clearly defining and effectively communicating the target audience, while prioritizing their reach, could enhance the intervention’s overall effectiveness.

Effectiveness

The results with respect to effectiveness suggest that the toolkit led to small, short-term improvements in knowledge on (sex differences in) IHD symptoms and cardiovascular risk factors, however, the high drop-out rate may have significantly affected the reliability of results. Moreover, the long-term effectiveness also remains uncertain, as such knowledge improvements may diminish over time [26]. To mitigate this, informational flyers and videos were distributed to reinforce the key information and support retention [19]. In line with previous research [27], focus group findings indicate that spaced message repetition is crucial for sustained symptom recognition. While repetition during the presentation is easily implemented, maintaining it over time may be more challenging and costly, requiring the organization of frequent community gatherings. Alternatively, instead of frequent repetition of the message, the solution may be sought in an easy-to-remember message or a mnemonic device [28]. A well-known example is the stroke awareness campaign F.A.S.T. (Face, Arm, Speech, Time), a multi-phase campaign that included television, press, and radio advertisements presenting a mnemonic device to promote recognition of stroke symptoms. This campaign was proven effective in promoting stroke symptom recognition and care seeking for stroke symptoms on the long term [29]. A similar mnemonic device might improve IHD symptom recognition. However, unlike stroke, which has a distinct symptom profile, IHD presents with more varied and less specific symptoms, making a direct translation of this approach less straightforward.

Beyond the effect on knowledge of IHD symptoms, the effect on care seeking behavior should be critically assessed. Increased knowledge of IHD symptoms and when to seek care could reduce the number of missed or delayed IHD diagnoses. However, referring back to the causal chain described under ‘intervention components’, we were only able to empirically assess short-term knowledge changes and intended care seeking, yet we were unable to assess the effect on actual care seeking. Moreover, unwanted side effects may occur, and should be assessed. For instance, given the various nature of potentially cardiac symptoms, and the high prevalence of these symptoms in the general population [30,31], large-scale information campaigns may spike the number of (unnecessary) healthcare visits for these symptoms [19]. This could pose a burden for the healthcare system, especially given the already high workload in Dutch general practice care [32].

Adoption

While we were unable to quantify findings on the adoption, we observed a high adoption rate of the intervention among the approached communities, partly due to interest in the topic of IHD. However, strong rapport between community leaders and researchers also played a key role, serving both as a motivation, and in some cases, a starting point for collaboration. This relates to the ethics of co-creation, referring to the invisible work done by participatory researchers around the ethical aspects of collaborating with patients or participants [33]. In particular, in communities that expressed more mistrust towards Dutch institutions [34], emotion- and relationship work was necessary to foster the necessary trust for collaboration [35]. This also required awareness of unequal power dynamics, and acknowledging how societal and systemic issues led to this mistrust. While preventing unequal power dynamics may be difficult in practice, there is significant attention for promoting equity in co-creation within the implementation science literature [36–38]. Literature describes various different models, frameworks, and theories for implementation scientists studying ethnicity-related health inequalities [37], and experts in this field created a charter which includes seven principles to promote equity in co-creation [36]. Our experiences align with these studies, as we found fostering trust, intentionally including a diverse range of individuals in the co-creation process, acknowledging ongoing and historical harms [36], overcoming implicit bias, and reflecting on one’s own positionality [38] were important prerequisites to promote equity in co-creation.

Moreover, in line with previous research, community engagement [39,40] and cultural-sensitivity of the intervention [40,41] were important facilitators for the adoption and effectiveness of the intervention. Attendees and community leaders specifically mentioned the co-creative approach taken during the development and implementation, the visual, narrative, interactive, and easy-to-understand presentation, the knowledgeable speaker with a similar cultural or ethnic background, and the free health checks. Overall, this led to high levels of satisfaction across all settings, while the acceptability of interventions is known to vary across cultural groups [42]. This suggests that the adaptability of the intervention may enhance both acceptability and social validity of the intervention across different settings [42].

Implementation

With respect to the implementation, facilitators included an interest in the topic of heart disease, and good rapport with community leaders and members for the collaboration. This good report and the collaboration with communities may be a double-edged sword, as this allowed us to share the organizational work with them, and it led to improved satisfaction and ownership amongst communities. One notable challenge to the implementation was infidelity to the intervention. While the adaptability of the toolkit was considered a great strength, including adapting the language and the content of the presentation to attendees’ cultural background [43], this adaptability may negatively affect fidelity to the intervention when intervention components that should not be adapted are altered (i.e., drift) [44], and could thus have affected the effectiveness of the intervention. Therefore, the fidelity to the toolkit should be safeguarded, to ensure that essential intervention components, in particular the medical information and presence of interactive elements, are not altered or excluded.

Other challenges to the implementation were related to organizational challenges, and a lack of clear communication about planning and finances. To prevent these challenges in future information sessions, we have now incorporated recommendations to improve the organization of sessions. This includes recommendations for clear communication (such as discussing finances and dividing responsibilities), and more practical recommendations (e.g., checking if the technological equipment is working, a checklist of all necessary resources) in the toolkit. Nevertheless, it is important to recognize that implementation of interventions in real-world settings means there is no absolute control over the circumstances, and therefore, some degree of flexibility on the organization’s behalf is required. However, a strength related to the development and implementation of this toolkit in real-world settings instead of experimental settings, means that the generalizability of the intervention can be considered high [45].

Maintenance

Despite the significant attention for early recognition of IHD at an (inter)national [46,47] level, and the high demand from communities for information sessions, the maintenance of the toolkit appeared to be challenging, as this required the integration of the intervention into existing infrastructure within a potential maintenance organization [42]. This required financial and material (e.g., for the rent of locations, audiovisual media, flyers) as well as human resources (presenters, organizers) for the organization of such live sessions. While budget was available within this study, finding and securing long-term funding for such activities is challenging. Moreover, maintenance of the intervention required experience with setting up such live community sessions and a large network among different ethnic minority groups. This is highly dependent on the network, experience, and organizational skills and capacity of the person in charge of organizing these sessions. This reliance on individual networks, skills, and experience may harm the sustainability and broader implementation of the intervention. We found these resources and this experience within the Municipal Health Services Haaglanden, the organization currently in charge of maintenance and scaling of the toolkit. Finally, in addition to determining the effectiveness, for potential maintenance organizations it would also be relevant to determine the costs associated with the implementation and maintenance, and these should be weighed against the expected effects of the intervention.

Conclusion

In conclusion, we were able to successfully co-create and pilot-implement a toolkit to promote IHD symptom recognition. The toolkit was well-received by attendees and community leaders, and findings concerning the reach, adoption and implementation were positive. The (cost)effectiveness should be further evaluated to study the long-term effect on symptom recognition, as well as the effect on care seeking. Moreover, the need for integration of the intervention in current infrastructure constitutes a challenge to the maintenance of the toolkit.