Engaging key populations, including gender and sexual minorities, is essential to meeting global targets for reducing new HIV infections and improving the HIV continuum of care. Negative attitudes toward gender and sexual minorities serve as a barrier to political will and effective programming for HIV health services. The President’s Emergency Plan for AIDS Relief (PEPFAR), established in 2003, provided Gender and Sexual Diversity Trainings for 2,825 participants including PEPFAR staff and program implementers, U.S. government staff, and local stakeholders in 38 countries. The outcomes of these one-day trainings were evaluated among a subset of participants using a mixed methods pre- and post-training study design. Findings suggest that sustainable decreases in negative attitudes toward gender and sexual minorities are achievable with a one-day training.
Citation: Poteat T, Park C, Solares D, Williams JK, Wolf RC, Metheny N, et al. (2017) Changing hearts and minds: Results from a multi-country gender and sexual diversity training. PLoS ONE 12(9): e0184484. https://doi.org/10.1371/journal.pone.0184484
Editor: Peter A. Newman, University of Toronto, CANADA
Received: December 27, 2016; Accepted: August 24, 2017; Published: September 19, 2017
This is an open access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication.
Data Availability: Ethical restrictions prevent us from being able to make the data from this study publicly available. We approached PEPFAR leadership to identify appropriate ways for a researcher to request and access the data. We were informed that the data set could not be made available for the following reasons: (1) The topic is highly sensitive in nature and there is a real potential for breach of confidentiality in countries where PEPFAR teams are small. (2) The GSD training evaluation was designed to inform PEPFAR of the efficacy of their intervention and was not designed as a research study for generalizability. The findings are shared here as a “proof of concept,” i.e. that large scale up of GSD trainings is feasible and can be effective. (3) As part of the informed consent process, participants were assured that their data would only be made public in aggregate so that no individual or country could be singled out. Providing access to the data set would violate this agreement with participants. Providing a data set without country level identifiers would preclude analyses that control for clustering by country – an important analytic step. Any questions regarding data may be sent to Ryan Olson at Ryan.Olson@thepalladiumgroup.com.
Funding: This publication was made possible by support from USAID and PEPFAR to the Health Policy Project (HPP) under Agreement No. AID-OAA-A-10-00067 and with assistance from the Johns Hopkins University Center for AIDS Research, an NIH funded program (P30AI094189), which is supported by the following NIH Co-Funding and Participating Institutes and Centers: NIAID, NCI, NICHD, NHLBI, NIDA, NIMH, NIA, FIC, NIGMS, NIDDK, and OAR. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.
Competing interests: The authors have declared that no competing interests exist.
Gender norms and gender inequities contribute to HIV vulnerability and are barriers to care and treatment. Endemic gender based violence, driven by deep inequalities, creates conditions where women are up to three times more likely than men to become infected with HIV . HIV disparities are even greater when examined by gender and sexual minority status. According to global meta-analyses, men who have sex with men (MSM) and transgender women (TW) experience 19 and 49 times the risk of HIV infection, respectively, compared to the general population of reproductive age adults [2, 3]. MSM and TW commonly face family rejection as well as social and cultural stigma and discrimination. These factors not only act as facilitators of the HIV epidemic but are often exacerbated by the criminalization of same-sex behaviors [4–6]. The effects of negative attitudes about gender equality and towards MSM, TW, and other gender and sexual minorities (GSM) held by policymakers (e.g. legislators and members of government ministries), health practitioners, and program staff are far reaching and contribute to the persistence of global HIV epidemics [7, 8]. For example, less than half of all countries included any reference to transgender people in their 2014 national HIV plans .
Gender- and sexuality-related stigma is often a hypothesized reason for the systematic global underinvestment in HIV programs that serve MSM and TW [8, 10–13]. Stigma-related marginalization of GSM is a likely explanation, given that data demonstrating the disproportionate burden faced by MSM around the world have been available for nearly two decades. The public health implications and human rights impact of laws and policies that perpetuate attitudes of stigma and discrimination need to be underscored as barriers to quality healthcare for MSM, TW and other GSM .
At service delivery levels, GSM’s negative experiences with health professionals erode trust and communication while seeking health services . Providers are rarely expected to gain competency in communicating with and treating GSM patients, resulting in mislabeling and maltreatment [15, 16]. Other times, GSM are turned away entirely; or, due to the fear of stigmatizing treatment, GSM delay seeking care [17, 18]. For example, recent evidence drawn from an online sample of 2,981 MSM confirms significant associations between anticipation of homophobia and not seeking HIV prevention services .
Research findings suggest that attitudes around the world toward GSM are changing. In a survey of 37,653 citizens of 39 countries, Horowitz et al. found that acceptance of homosexuality increased by at least five percentage points in one-third of countries between 2007 and 2013 . Still, more than 60 percent of respondents in El Salvador and Kenya and over 90 percent of respondents in Indonesia, Uganda, Ghana, and Nigeria indicated in 2013 that they believed homosexuality should not be accepted by society.
Increasing acceptance of GSM among policymakers, program staff, and health providers is a crucial step toward ensuring better access and uptake of HIV services. According to leading social psychologists, Zanna and Rempel, attitudes are the product of cognition (i.e., information and beliefs), affect (i.e., emotions), and past experiences [21, 22]. Efforts to create short-term changes in attitudes by way of sensitization workshops or training have been shown to result in positive changes with moderate consistency [23–26]. However, the effects of interventions on attitudes typically decay over time . The distinction between short-term and long-term changes after such interventions is important for understanding how frequently training may be needed. Yet, long-term changes, also referred to as the attitude durability, are rarely measured experimentally.
The purpose of this manuscript is to report on the evaluation of the Gender & Sexual Diversity (GSD) Training that was developed by the Health Policy Project (HPP), an entity funded via the United States Agency for International Development (USAID) and the President’s Emergency Plan for AIDS Relief (PEPFAR, established in 2003). In coordination with a U.S. government interagency team comprised of members of the PEPFAR Key Populations Working Group and the PEPFAR Gender Technical Working Group, HPP offered the required training to PEPFAR-funded country teams. The curriculum was designed to help country programs understand and address the needs of GSM communities in the context of HIV programming and U.S. workplace policies on non-discrimination. Funding for the training was provided by PEPFAR via USAID.
With the recognition that addressing norms and attitudes related to gender and sexual orientation are important components of gender integration, the Gender & Sexual Diversity training focused on how gender and sexuality affect all people. The training aimed to (1) educate participants on terminology, U.S. government policies, and workplace expectations related to gender and sexual diversity; (2) sensitize participants to the needs of GSM, including as beneficiaries of HIV and other health programs; (3) provide examples and recommendations for meaningfully engaging GSM beneficiaries when developing and implementing programs; and (4) connect participants with local and regional resources on gender and sexual diversity issues. Using Zanna and Rempel’s conceptualization of attitudes, the training modules were designed to foster attitude change by impacting cognition and affect, and linking those to past experiences . The purpose of the evaluation was to assess the training’s efficacy in changing attitudes toward GSM and determine its value as a tool to foster more effective key population programming.
The GSD Training curriculum was facilitated over the course of one full work day and contained four modules. Each module was designed to be highly engaging, employing diverse pedagogies that included slide presentations, interactive exercises, discussions and break out groups, and confidential electronic audience response systems—all with the intent of addressing components that shape attitudes (i.e., cognitions, affect, and past experiences).
Modules 1 and 2 provided information designed to impact cognition and opportunities to reflect on past experiences. Module 1 topics included: (1) epidemiological, financial, and human rights arguments for reaching and engaging GSM and increasing access to appropriate health services; (2) existing legal protections for U.S. government staff against employment discrimination based on sexual orientation and gender identity; (3) gender norms and their impact on health; and (4) the Updated PEPFAR Gender Strategy. Module 2 introduced concepts and terminology such as biological sex, gender expression, gender identity, and sexual orientation, and included activities that dealt with language, discrimination, and personal advocacy. Module 3 was focused on affect and included a panel of two members of local LGBT communities and one human rights advocate who each shared their story, describing what they believed to be the key challenges, success stories, and priorities for engaging with GSM in that country. Module 4 featured a discussion to build a shared understanding of the importance of meaningful engagement with GSM and an activity to illustrate meaningful engagement in action. This module was designed to build on potential positive attitude changes and promote positive behavioral intentions .
The GSD training was developed by context experts and training specialists at HPP in collaboration with members of the PEPFAR Key Populations Working Group and the PEPFAR Gender Technical Working Group. It was designed specifically for PEPFAR staff and their country-level implementing partners. This diverse audience spanned nationalities, languages, generations, religious and political beliefs, as well as educational backgrounds. Particular care was given to developing a curriculum that was flexible enough to be used across cultures.
While the curriculum remained the same in each country to ensure consistent messaging, facilitators met with local PEPFAR staff and local GSM community members prior to each training to discuss and prepare for the specific country context. GSM community members served as panelists during Module 3, during which they presented on personal and professional experiences of GSD in their country. When possible, GSM community members attended the entire training, joining other participants in conversations about GSD concepts and meaningful engagement. The final training manual and materials are available online at http://www.healthpolicyproject.com/index.cfm?ID=GSDTraining.
This independent outcome evaluation was approved by the ethics review board at Health Policy Project. All data were received anonymously for analysis by TP, CP, and BASN. The study was conducted using an embedded mixed methods pre-test/post-test design (Fig 1) in which qualitative data were collected to supplement the quantitative data . Quantitative survey data were collected immediately before, immediately after, and three months following the GSD Training. Qualitative data were collected three to six months after the training to provide depth and context for the quantitative results.
The GSD Training was offered to PEPFAR staff and program implementers in 38 countries. Trainings were open to other interested U.S. government staff and local stakeholders at the discretion of each PEPFAR country team. Each training lasted one day; and trainings were held multiple times within each country in order to accommodate the maximum number of participants. A total of 2,825 people in 38 countries received the PEPFAR GSD Training. Of these, 2,715 individuals in 33 countries participated in the training where a formal course evaluation was possible. Among these individuals, 1,766 people provided informed consent and valid data for a participation rate of 65%. Sample sizes for each included country are listed in Table 1. Data were excluded from five countries where trainings took place: Cameroon, Cote d’Ivoire, Honduras, Haiti, and Democratic Republic of Congo (DRC). In Cameroon, only pre-test data were collected using a pilot version of the questionnaire. In Cote d’Ivoire and Honduras, technical difficulties precluded data collection; and translation inconsistencies between pre- and post-test questionnaires invalidated data from Haiti and DRC.
Prior to data collection activities, participants were asked to provide informed consent for their de-identified data to be used for evaluation. Data from training participants who did not provide consent were destroyed and excluded from analyses. Training participants used individual handheld devices to indicate consent and to complete a brief set of questions measuring attitudes towards GSM and gender variance immediately prior to the training and immediately after the training. The devices transmitted data directly into Turning Point 5 software (Turning Technologies, Youngstown, OH) without being visible to participants. Use of individual handheld devices allowed for confidential linking of pre- and post-training data. Due to technology limitations, training participants in Barbados and Nicaragua used paper versions of the pre- and post-training questionnaires which were later manually entered into the software. Participants who consented to the use of their data recorded their participant number on each questionnaire so that data could be linked. Data were electronically stored on a password, encrypted cloud-based folder that was maintained by HPP. These data storage procedures ensured that the assessment information remained confidential and that only authorized personnel had access to it. Evaluation results are presented as aggregated data and no individual information is shared. While the PEPFAR training was required, consenting to have responses analyzed (i.e., participate in the evaluation) was a voluntary process.
Three months after the training, participants who provided consent were sent an email with a personalized link to the follow-up questionnaire. One reminder email was sent to non-responders approximately one month after the initial email with a second reminder being sent no less than two weeks apart for those who had not responded. Of the 1,766 initial participants, 576 (33%) responded to the 3-month follow-up survey. At no time did the evaluator have access to any identifying information for any of the participants. The evaluation was determined to be exempt by the ethical review committee at Palladium and a non-research determination (NRD) was approved by the Centers for Disease Control and Prevention (CDC).
Demographic data included gender (men, women, neither/both), age, and professional role (PEPFAR staff, non-PEPFAR U.S. government staff, program implementers, and other). Attitudes toward GSM were measured using an 8-item scale adapted from the 25-item scale, Support for Lesbian and Gay Human Rights . The new scale had high internal consistency with a Cronbach alpha of 0.82. Attitudes toward gender variance were measured using a 6-item scale adapted from the 32-item Gender and Transgenderism Scale . This new scale was also internally consistent with a Cronbach alpha of 0.83. Exploratory factor analysis indicated that a two-factor solution fit the data well; and item loadings were consistent with the two identified scales. Both measures used a 5-point Likert scale with response options ranging from strongly agree (1) to strongly disagree (5). Items for which agreement indicated a more negative attitude were reversed coded for aggregate analyses. Higher scale scores represented more negative attitudes toward GSM and gender variance.
In addition to these scales, participants were asked two questions to assess their self-efficacy for working with GSM: “I can list several ways I could take action to reduce stigma and discrimination against gender and sexual minorities in my workplace;” and “I can name at least two local organizations OR activists in my country that are supporting the health and human rights of gender and sexual minorities.” The number of measures that could be included was limited by the need to ensure that data collection was brief (i.e. only a few minutes) and did not delay or prolong the training activities.
Data were analyzed using STATA version 13.1 (StataCorp, College Station, TX) and reported only for GSD trainees who provided informed consent. Frequencies were calculated for each of the demographic characteristics. For each of the scale items, the proportion of participants who agreed and strongly agreed was calculated. Next, for each participant, the overall score for each scale was taken as the average of the individual questions (after reverse coding of appropriate items). Where possible, the pre-training and post-training scores were linked by individual unique participant identifiers. Not every person who completed the pre-training survey remained for the post-training survey; thus, the sample size for the post-training scores is lower than for the pre-training scores. In addition, not every respondent to the emailed follow-up survey provided their participant identifier. Therefore, it was not possible to link data for all participants at all three time periods. The average scale scores were used to compare differences across time periods. The paired student t-test was used to determine statistical significance at p-values ≤ 0.05 for comparisons between pre- and post-training scores. The loss of more than two-thirds of the sample between pre-training (n = 1766) and follow-up (n = 576) precluded meaningful statistical comparison of the aggregate data at these two time points. However, a sub-analysis was conducted of data from the participants (n = 211) who could be linked at pre-training, post-training, and follow-up.
We fitted ordinal logistic regression models with random effects for participants and countries, and adjusted for participants’ age and gender . These models were applied to the linked pre- and post-training data. Additionally, we applied the same models to data from the 211 participants who could be linked across all three time points. The distribution of the missing data across countries, age, and gender indicated that missing data were randomly distributed across these variables. We therefore conducted the regression analyses assuming responses were missing at random.
Follow-up surveys completed three months after the training included an open-ended question asking respondents to describe any changes they had made in their work which may have been related to participation in the training. In addition, 29 qualitative key informant interviews were conducted in 21 countries approximately three to six months after the training. These interviews were designed to identify any perceived changes in the PEPFAR country team’s relationship with GSM communities after participation in the training. Key informants were purposively selected based on engagement in the training and familiarity with GSD issues. All key informants had attended the GSD training in their country, either as a community panelist or PEPFAR participant. For each country, the GSD training team identified a community panelist who had a strong understanding of the community and were present and engaged during the training. Each GSD training team also identified a PEPFAR staff member who attended the training and was engaged and familiar with the topic of GSD in the context of that country. These 29 individuals verbally consented to participate in the key informant interviews.
Twenty-one interviews were conducted with community members who had participated on the panel during the training. Eight interviews were conducted with PEPFAR staff who attended the training. Staff turnover and busy schedules limited availability of PEPFAR staff for follow-up interviews.
Key informant interviews were conducted by Skype or telephone and lasted approximately 30 minutes. A semi-structured guide was used to organize the interview. Community members were asked to describe their role in the community and their relationship with PEPFAR, including receipt of PEPFAR funding, and to discuss whether they noticed any differences in local PEPFAR programming or community engagement after the training. PEPFAR staff were asked to describe their programming, in particular with key populations (i.e., MSM, transgender persons, people who inject drugs, and sex workers), and to discuss whether they noticed any differences in the workplace after the GSD Training. Interviews were led by trained staff and detailed notes were taken by a separate note-taker. Post interview field notes were typed by both the interviewer and note-taker.
Thematic analysis was used for the qualitative data . The interview notes and short, open-ended responses were analyzed separately by two coders who did not participate in any data collection activities. Each coder read all of the qualitative data for recurring themes and patterns, then sorted the text by theme. Memos were used to track the analytic process. Any differences in noted themes were discussed with the lead investigator until consensus was reached. Themes were then grouped by topical area based on research questions described above.
Participant demographics are listed in Table 2. Of the 1,766 participants in the study, 1,738 (98%) had responses for the pre-training survey and 1,429 (81%) contributed responses to the post-training survey. Of the 1,738 respondents with pre-training survey data, 46% (n = 790) were aged 39 years or younger, 48% (n = 841) were 40 years or older and 6% (n = 107) were missing age data. Fifty-three percent (n = 916) identified as women, 39% (n = 670) identified as men, 3% (n = 52) selected “neither/both” and 6% (n = 100) were missing gender data. Only 1,638 participants provided data on professional role. Most respondents were PEPFAR staff (53%, n = 892) or program implementers (32%, n = 539). The demographic distribution of the participants in the follow-up survey did not change.
Attitude changes by item
Overall, attitudes toward GSM and gender variance were more positive after the training compared to before the training. Item analysis within each scale revealed variability in the amount of change from baseline to immediately post-training (Table 3). The largest change in attitude toward GSM took place in responses to the following two questions: 1) “All people should be able to have any kind of consensual sex in private without being fined or arrested.” Agreement with this statement rose from 73% to 84%. 2) “Gender and sexual minorities should be allowed to express their opinions in public as long as they don’t offend most people.” Agreement with this statement rose from 65% to 75%. The vast majority of participants before (91%) and after (95%) the training agreed that no one should experience job discrimination because of their sexual orientation. Agreement that same sex couples should be allowed to marry was reported by 43% of participants prior to completing the training; and 48% agreed with this statement after the training. The least amount of change was seen in response to the statement: “It is okay for a newspaper to publicize that a person is a gender or sexual minority without that person’s permission.” Eight percent of participants agreed with the statement before the training; while 6% agreed after the training.
The largest change in attitude toward gender variance was seen in response to the question about whether people are either men or women (intended to imply a denial of non-binary gender identities), with a decrease from 56% to 37% who agreed with this statement. The majority of participants were comfortable with masculine women before (63%) and after (71%) the training. A smaller proportion of participants were comfortable with feminine men as compared to masculine women, both before (57% vs 63%, respectively) and after (67% vs 71%, respectively) the training. However, most participants were comfortable working with feminine men before (72%) and after (80%) the training. The smallest proportion of participants agreed that a man should be able to dress like a woman, both before (49%) and after (61%) the training.
Attitude change by participant characteristic and scale scores
Table 4 depicts aggregate scale scores at baseline, immediately after the training, and at the 3-month follow-up period. Baseline scores were higher (indicating more negative attitudes) towards gender variance as compared to gender and sexual minorities (mean score 2.7 and 2.1, respectively). In the unadjusted, bivariate comparisons depicted in the table, overall scale scores demonstrated significantly less negative attitudes toward both GSM and gender variance post-training compared with pre-training. This change was consistent across gender and age. The same or lower scores were observed in both scales when comparing post-training and follow-up results.
On multivariable analyses adjusted for age and gender, the odds of higher scores (i.e. more negative) on the Attitudes Toward GSM scale were reduced by 73% from pre- to post-training [OR 0.27; 95%CI: 0.23, 0.33, p<0.0001] (Table 5: Model A). In this model, women had less negative attitudes than men [OR 0.56; 95%CI: 0.38, 0.82, p = 0.003], but there were no significant differences in scores by age. In an adjusted model that included only participants with linked data at all three time points (Table 5: Model B), the odds of higher scores on the Attitudes Toward GSM Scale were reduced by 76% from pre- to post-training [OR 0.24; 95%CI: 0.15, 0.39, p<0.0001] and 71% from pre-training to 3-month follow-up [OR 0.29, 95%CI: 0.18; 0.46, p<0.0001]. In this adjusted model, participants aged 40 years and older were more likely to have higher scores compared with those 39 and younger [OR 3.07; 95%CI: 1.2, 8.41, p = 0.029]; however, there were no significant differences in scores by gender.
Similarly, Attitudes Toward Gender Variance scale scores were lower after the training compared with pre-training scores. On multivariable analyses adjusted for age and gender, the odds of higher scores (i.e. more negative attitudes toward gender variance) were reduced by 71% from pre- to post-training [OR 0.29; 95%CI: 0.24, 0.34, p<0.0001] (Table 6: Model A). In this model, women had lower scores (less negative attitudes) than men [OR 0.38; 95%CI: 0.26, 0.55, p<0.0001]; and older participants had higher scores (more negative attitudes) than younger ones [OR 1.49; 95%CI 1.04, 2.15, p = 0.031]. In an adjusted model with all three time points (Table 6: Model B), the odds of higher scores were reduced by 68% from pre- to post-training [OR 0.32; 95%CI: 0.21, 0.48, p<0.0001] and 79% from pre-training to 3-month follow-up [OR 0.21, 95%CI: 0.14; 0.33, p<0.0001]. There were no significant differences by age or gender.
Sensitivity analyses were conducted by fitting regression models that excluded participants with missing data at either pre- or post-training, and the results did not change significantly. In addition, analyses were conducted excluding participants who answered less than half of the questions on each scale. Again, the results did not change significantly.
Changes in self-efficacy and self-reported behavior
The largest change from baseline took place in self-efficacy measures (Table 3). Self-efficacy for reducing GSM stigma and discrimination in the workplace rose from 73% to 95% post-training and remained high at 95% three months after the training (n = 568, data not shown). The proportion of participants who could identify local organizations or activists who support GSM human rights increased from 69% to 88% and was maintained at 87% three months after the training (n = 573, data not shown). These findings were compared with the subset of participants for whom linked pre-, post-, and 3-month follow-up data were available for these items (n = 192). No differences were found with the following exception: 96% of this subset of participants were able to identify local organizations or activists immediately after the training and 89% were able to do so three months following the training.
Approximately half of the respondents (53%) on the follow-up survey reported having done something differently in their workplaces or programs following the GSD training; and 311 provided specific comments about what they had done differently. Participants described actions that fell into three broad categories in order of frequency: (1) treating GSM with greater acceptance or respect; (2) educating others; and (3) adapting or modifying GSM programming. Table 7 includes illustrative quotes from participant responses.
Qualitative findings on community engagement and programming
Qualitative interviews allowed participants to describe their experience of the training and any changes they noticed in the PEPFAR teams’ work with GSM as a consequence of the training. Every person interviewed (n = 29) described the community panel as the most compelling part of the training. Community members felt empowered by their experiences on the panel, and PEPFAR staff felt that people connected with the experiences of GSM in a deeper way after interacting with panelists. A community member stated, “So that it’s not just top down, it was bottom-up. It was an opportunity for meaningful engagement to take place.” A PEPFAR staff member agreed, “I think the panel was really a highlight in having an opportunity to see that these are real people and they aren’t being pushed to the side.”
Most PEPFAR staff and all community members interviewed felt that the training was valuable and recommended that it take place regularly and reach a wider audience. However, few could link the training to a specific change in PEPFAR community engagement or programming for GSM. Several PEPFAR respondents pointed out that the training took place after their annual planning cycle; therefore, it would be another year before they would know if there was an impact. Some noted that 3–6 months was not enough time to determine if there had been a change. Other PEPFAR staff felt that their team was already appropriately engaging GSM in their work; therefore, changes weren’t needed. A few PEPFAR staff members noted that meeting the panelists provided an opportunity to engage with a community organization that the team had not known before. The majority of community members interviewed had little familiarity with the PEPFAR program planning process. However, a few community members noted new or more engaged relationships with PEPFAR staff or implementers in the 6 months since the training. For example, a community member stated, “Previously there were organizations that partners would not engage with but are now engaging with. The training itself opened doors to different partners that we work with. These people are resources. The training itself is something that made people think out of the box.”
The Joint United Nations Programme on HIV/AIDS (UNAIDS) has set ambitious targets to end the AIDS epidemic by 2030. These targets include ensuring that 90% of people living with HIV know their HIV status; 90% of people living with HIV receive antiretroviral therapy (ART); and 90% of people receiving ART have viral suppression . In support of these “90-90-90” targets, PEPFAR 3.0 committed to a human rights agenda that focuses on ending stigma and discrimination against key populations, including GSM, in order to reach them with HIV testing, prevention, and treatment services .
Effective engagement of GSM in HIV services requires a greater understanding and acceptance of GSM among policymakers, program implementers, and service providers. Both qualitative and quantitative findings at post and follow-up assessments of this one-day PEPFAR GSD training indicate improvements in attitudes toward GSM as compared to baseline findings. Qualitative data suggest that many of the training participants translated these positive attitude changes into action in the workplace.
By challenging negative beliefs about GSM with factual information, emotionally engaging participants during the community panel, and encouraging positive behavioral intentions through role plays and practical exercises, the PEPFAR GSD training addressed key components of attitude formation . A pre-training to post-training decrease in negative attitudes was not surprising for a one-day training that received high level support from in-country leadership and in which facilitators set clear norms for GSD respect and tolerance. However, the effects of attitude change interventions typically decay with time . Unlike most interventions, more positive attitudes toward GSM not only persisted 3–6 months after that training but improved even further. It is possible that attendance by most, if not all, PEPFAR staff at a site may have helped to reset the norms of the workplace, setting up a self-reinforcing environment of tolerance and expectation of respect.
Several study limitations exist. The evaluation had a low response rate (33%) to the three-month follow-up questionnaire. However, this response rate falls within the range of 24–48% that is typical of many email surveys [37, 38]. A limited subsample (12%) provided data that could be linked at all three time points (pre-, post-, and 3–6 months following the training); and we observed improvements in attitudes within this sub-sample that exceeded those of the overall three-month respondents. There were also a limited number of PEPFAR staff interviews even though they were the intended audience of training. The possibility that those who did not respond had the most negative attitudes must also be considered; and generalizability of the finding are limited. While the subsample with linked data at all three time points may not represent the full sample, the findings suggest the possibility of an enduring positive change in attitudes towards GSM. The study was limited to self-reported data. We were unable to verify the behavioral changes reported by participants in the qualitative data, and our measures for self-efficacy had not been validated in prior studies. The need to limit the questionnaires to no more than five minutes precluded the inclusion of a social desirability scale. Therefore, we cannot assess how much the results may have been impacted by social desirability bias. Future studies would benefit from assessing this important source of bias.
Despite limitations, triangulation of all data sources (attitude scales, open-ended survey responses, and key informant interviews) suggests that the one-day PEPFAR GSD training was successful in changing attitudes toward GSM. Importantly, these changed attitudes were sustained for months after the training, and attendees noted positive changes in the workplace and, to a lesser extent, HIV programming. It is likely that repeated exposure via annual or semi-annual refresher trainings would be needed to sustain positive attitudes long-term as PEPFAR staff come and go based on the rotating nature of U.S. government international positions. As the first study to report GSD attitude change among a global sample of almost 1,800 individuals from 33 countries, the findings provide strong evidence for the capacity of carefully crafted GSD trainings to effect change. More longitudinal research is needed to better understand how long these changes in attitude may persist and to determine the appropriate intervals for refresher trainings. Given the great time demands on PEPFAR staff and implementers, future research should also explore whether alternative training modalities (e.g., shorter trainings, online trainings) would have similar outcomes.
The findings and conclusions in this article are based upon the Gender & Sexual Diversity Training which was developed by the USAID- and PEPFAR-funded Health Policy Project, in coordination with a U.S. government interagency team, including members of the PEPFAR Key Populations Technical Working Group and the PEPFAR Gender Technical Working Group. We are grateful to the dedicated staff who devoted countless hours to developing and implementing the GSD Training as well as the numerous training participants for their willingness to take part in this evaluation.
- 1. World Health Organization. Violence Against Women and HIV/AIDS: Critical Intersections—Intimate Partner Violence and HIV/AIDS. Geneva: World Health Organization, 2017.
- 2. Baral S, Beyrer C, Muessig K, Poteat T, Wirtz AL, Decker MR, et al. Burden of HIV among female sex workers in low-income and middle-income countries: a systematic review and meta-analysis. Lancet Infect Dis. 2012;12(7):538–49. Epub 2012/03/20. pmid:22424777.
- 3. Poteat T, Reisner SL, Radix A. HIV epidemics among transgender women. Curr Opin HIV AIDS. 2014;9(2):168–73. pmid:24322537.
- 4. Wade AS, Kane CT, Diallo PA, Diop AK, Gueye K, Mboup S, et al. HIV infection and sexually transmitted infections among men who have sex with men in Senegal. AIDS. 2005;19(18):2133–40. pmid:16284463.
- 5. Soto RJ, Ghee AE, Nunez CA, Mayorga R, Tapia KA, Astete SG, et al. Sentinel surveillance of sexually transmitted infections/HIV and risk behaviors in vulnerable populations in 5 Central American countries. J Acquir Immune Defic Syndr. 2007;46(1):101–11. pmid:17972366.
- 6. Chan PA, Khan OA. Risk factors for HIV infection in males who have sex with males (MSM) in Bangladesh. BMC Public Health. 2007;7:153. pmid:17625020; PubMed Central PMCID: PMCPMC1931591.
- 7. Stahlman S, Hargreaves JR, Sprague L, Stangl AL, Baral SD. Measuring Sexual Behavior Stigma to Inform Effective HIV Prevention and Treatment Programs for Key Populations. JMIR public health and surveillance. 2017;3(2):e23. Epub 2017/04/28. pmid:28446420; PubMed Central PMCID: PMCPMC5425775.
- 8. Grosso A, Ryan O, Tram KH, Baral S. Financing the response to HIV among gay men and other men who have sex with men: Case studies from eight diverse countries. Global Public Health. 2015;10(10):1172–84. pmid:26139083
- 9. UNAIDS. National Commitments and Policies Instrument 2014 [cited 2016 May 10]. Available from: http://www.unaids.org/en/dataanalysis/knowyourresponse/ncpi/2014countries.
- 10. Beyrer C. Hidden yet happening: the epidemics of sexually transmitted infections and HIV among men who have sex with men in developing countries. Sexually transmitted infections. 2008;84(6):410–2. pmid:19028936.
- 11. Trapence G, Collins C, Avrett S, Carr R, Sanchez H, Ayala G, et al. From personal survival to public health: community leadership by men who have sex with men in the response to HIV. Lancet. 2012;380(9839):400–10. pmid:22819662; PubMed Central PMCID: PMCPMC3805044.
- 12. Reynolds R, editor Social discrimination against men who have sex with men (MSM) implications for HIV policy and programs. Global Forum on MSM & HIV; 2010.
- 13. Wirtz A, Trapence G, Jumbe V, Umar E, Ketende S, Kamba D, et al., editors. HIV prevalence, sexual risks and HIV knowledge among MSM in Malawi: understanding risks among a stigmatized population and opportunities for interventions (Oral Presentation). XIX International AIDS Conference; 2012; Washington DC,.
- 14. Levy ME, Wilton L, Phillips G 2nd, Glick SN, Kuo I, Brewer RA, et al. Understanding structural barriers to accessing HIV testing and prevention services among black men who have sex with men (BMSM) in the United States. AIDS and behavior. 2014;18(5):972–96. pmid:24531769; PubMed Central PMCID: PMCPMC4509742.
- 15. O'Hanlan KA, Cabaj RP, Schatz B, Lock J, Nemrow P. A Review of the Medical Consequences of Homophobia with Suggestions for Resolution. Journal of the Gay and Lesbian Medical Association. 1(1):25–39.
- 16. Mansh M, Garcia G, Lunn MR. From patients to providers: changing the culture in medicine toward sexual and gender minorities. Acad Med. 2015;90(5):574–80. pmid:25650825.
- 17. Bateman C. Transgender patients sidelined by attitudes and labelling. S Afr Med J. 2011;101(2):91–3. pmid:21678727.
- 18. Lane T, Mogale T, Struthers H, McIntyre J, Kegeles SM. "They see you as a different thing": the experiences of men who have sex with men with healthcare workers in South African township communities. Sexually transmitted infections. 2008;84(6):430–3. pmid:19028941; PubMed Central PMCID: PMCPMC2780345.
- 19. Santos GM, Beck J, Wilson PA, Hebert P, Makofane K, Pyun T, et al. Homophobia as a barrier to HIV prevention service access for young men who have sex with men. J Acquir Immune Defic Syndr. 2013;63(5):e167–70. pmid:24135782.
- 20. Horowitz JM, Simmons K, Poushter J, Ponce A, Barker C, Devlin K. The Global Divide on Homosexuality. Pew Global Attitudes Project. 2013:1–26.
- 21. Huskinson TL, Haddock G. Individual differences in attitude structure and the accessibility of the affective and cognitive components of attitude. Social Cognition. 2006;24(4):453.
- 22. Zanna MP, Rempel JK. Attitudes: A new look at an old concept. 1988.
- 23. Prakashkumar MD, Ebenezer M, Richard J. Measurement of Change in the Knowledge and Attitude about Leprosy in Physiotherapy Students Undergoing Intensive One Week Training in Leprosy. Indian J Lepr. 2014;86(3):99–104. pmid:26118221.
- 24. Vincent M, Bartley G, Clearie A. Attitude and knowledge change in response to sexuality education training. Family life educator. 1985;3(4):23. pmid:12314199
- 25. Hurtado SL, Simon-Arndt CM, McAnany J, Crain JA. Acceptability of mental health stigma-reduction training and initial effects on awareness among military personnel. Springerplus. 2015;4:606. pmid:26543741; PubMed Central PMCID: PMCPMC4628072.
- 26. Varas-Diaz N, Neilands TB, Rodriguez-Madera SL, Padilla M. The role of emotions in the reduction of HIV/AIDS stigma among physicians in training. AIDS Care. 2016;28(3):376–83. pmid:26444133; PubMed Central PMCID: PMCPMC4747826.
- 27. Cook TD, Flay BR. The persistence of experimentally induced attitude change. Advances in experimental social psychology. 1978;11:1–57.
- 28. Haddock G, Zanna MP, Esses VM. Assessing the structure of prejudicial attitudes: The case of attitudes toward homosexuals. Journal of Personality and Social Psychology. 1993;65(6):1105–18.
- 29. Ajzen I. The theory of planned behavior. Organizational behavior and human decision processes. 1991;50(2):179–211.
- 30. Creswell JW, Clark VLP. Designing and conducting mixed methods research. 2007.
- 31. Ellis S, C K, Wilkinson S. Attitudes Towards Lesbians and Gay Men and Support for Lesbian and Gay Human Rights Among Psychology Students. Journal of homosexuality. 2003;44(1):121–38.
- 32. Hill DB, Willoughby BLB. The Development and Validation of the Genderism and Transphobia Scale. Sex Roles. 2005;53(7/8):531–44.
- 33. Sullivan GM, Artino AR Jr. Analyzing and interpreting data from likert-type scales. Journal of graduate medical education. 2013;5(4):541–2. Epub 2014/01/24. pmid:24454995; PubMed Central PMCID: PMCPMC3886444.
- 34. Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Research in Psychology. 2006;3(2):77–101.
- 35. UNAIDS. 90-90-90: An ambitious treatment target to help end the AIDS epidemic 2014 [cited 2016 March 25]. Available from: http://www.unaids.org/sites/default/files/media_asset/90-90-90_en_0.pdf.
- 36. U.S. Department of State. PEPFAR 3.0 Controlling the Epidemic: Delivering on the Promise of an AIDS-free Generation 2014. Available from: http://www.pepfar.gov/documents/organization/234744.pdf.
- 37. Baruch Y, Holtom BC. Survey response rate levels and trends in organizational research. Human Relations. 2008;61(8):1139–60.
- 38. Sheehan KB. E-mail Survey Response Rates: A Review. Journal of Computer-Mediated Communication. 2001;6(2):0–.