Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Measuring stigma affecting sex workers (SW) and men who have sex with men (MSM): A systematic review

  • Alanna Fitzgerald-Husek ,

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Software, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

    alanna.fitzgerald-husek@medportal.ca

    Affiliation Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada

  • Michael J. Van Wert,

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Software, Validation, Writing – original draft, Writing – review & editing

    Affiliation Community Psychiatry Program, Johns Hopkins Bayview Medical Center, Baltimore, Maryland, United States of America

  • Whitney F. Ewing,

    Roles Data curation, Formal analysis, Investigation, Project administration, Software, Validation, Visualization, Writing – original draft, Writing – review & editing

    Affiliation Centre for Public Health and Human Rights, Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Johns Hopkins University, Baltimore, Maryland, United States of America

  • Ashley L. Grosso,

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Supervision, Validation, Writing – original draft, Writing – review & editing

    Affiliation Centre for Public Health and Human Rights, Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Johns Hopkins University, Baltimore, Maryland, United States of America

  • Claire E. Holland,

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Writing – review & editing

    Affiliation Centre for Public Health and Human Rights, Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Johns Hopkins University, Baltimore, Maryland, United States of America

  • Rachel Katterl,

    Roles Data curation, Formal analysis, Investigation, Validation, Writing – original draft, Writing – review & editing

    Affiliation HIV/AIDS, STIs & Viral Hepatitis Programme, World Health Organization Regional Office for Europe, Copenhagen, Denmark

  • Lori Rosman,

    Roles Conceptualization, Data curation, Methodology, Software, Validation, Writing – review & editing

    Affiliation Welch Medical Library, Johns Hopkins School of Medicine, Johns Hopkins University, Baltimore, Maryland, United States of America

  • Arnav Agarwal,

    Roles Data curation, Investigation, Writing – review & editing

    Affiliations School of Medicine, University of Toronto, Toronto, Ontario, Canada, Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada

  • Stefan D. Baral

    Roles Conceptualization, Funding acquisition, Methodology, Project administration, Resources, Supervision, Validation, Writing – review & editing

    Affiliation Centre for Public Health and Human Rights, Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Johns Hopkins University, Baltimore, Maryland, United States of America

Measuring stigma affecting sex workers (SW) and men who have sex with men (MSM): A systematic review

  • Alanna Fitzgerald-Husek, 
  • Michael J. Van Wert, 
  • Whitney F. Ewing, 
  • Ashley L. Grosso, 
  • Claire E. Holland, 
  • Rachel Katterl, 
  • Lori Rosman, 
  • Arnav Agarwal, 
  • Stefan D. Baral
PLOS
x

Abstract

Background

Stigma involves discrediting a person or group based on a perceived attribute, behaviour or reputation associated with them. Sex workers (SW) and men who have sex with men (MSM) are key populations who are often at increased risk for the acquisition and transmission of HIV and who are affected by stigma that can negatively impact their health and well-being. Although stigma was included as an indicator in the US National HIV/AIDS Strategic Plan and there have been consultations focused on adding a stigma indicator within PEPFAR and the Global Fund in relation to potentiating HIV risks among key populations, there remains limited consensus on the appropriate measurement of SW- or MSM-associated stigma. Consequently, this systematic review summarizes studies using quantitative, qualitative, or mixed methods approaches to measure stigma affecting sex workers and men who have sex with men.

Methods and findings

This systematic review included English, French, and Spanish peer-reviewed research of any study design measuring SW- or MSM-associated stigma. Articles were published from January 1, 2004 to March 26, 2014 in PsycINFO, PubMed, EMBASE, CINAHL Plus, Global Health, and World Health Organization Global Health Library Regional Indexes.

Of the 541 articles reviewed, the majority measured stigma toward MSM (over 97%), were conducted in North America, used quantitative methods, and focused on internalized stigma.

Conclusions

With the inclusion of addressing stigma in several domestic and international HIV strategies, there is a need to ensure the use of validated metrics for stigma. The field to date has completed limited measurement of stigma affecting sex workers, and limited measurement of stigma affecting MSM outside of higher income settings. Moving forward requires a concerted effort integrating validated metrics of stigma into health-related surveys and programs for key populations.

Introduction

There is increasing interest and understanding of the adverse health outcomes associated with stigma [1]. Stigma involves marking and discrediting an individual or group on the basis of a real or perceived attribute, behaviour or membership to a group [1] and has been linked with negative outcomes at multiple levels. For individuals, studies have consistently found that stigma can result in lower self-esteem, poor academic achievement [2], and decreased uptake of health and social services [3]. At a social or macro level, stigma may influence legislation, policy decisions, insurance determinations, employment discrimination, and the orientation of research and theory [4, 5].

Of particular importance to those working in public health and health care policy and service delivery are the impacts of stigma on individuals’ mental and physical wellbeing. Among people living with HIV (PLHIV), studies show that higher stigma is associated with depression [68], anxiety [8], increased suicidality [9] and lower quality of life [10]. Higher stigma is also associated with a greater likelihood of chronic pain, poorer physical capacity [11], and morbidity related to lower levels of medication adherence [1214]. Stigma may also influence health through mediators including lessened resourcefulness, negative effects on social relationships, and contributing to high stress levels for the affected individual [15].

People stigmatize others based on a series of social constructs, which vary across time and cultures [16]. Despite this, some groups, identities, and behaviours are consistently stigmatized across much of the world. Examples include stigma based on: sexual practices and identities of gay men and other men who have sex with men (MSM); occupationally-linked behaviours and identities of sex workers (SW); individuals who are transgender; substance use and addictions among people who use drugs; and health status of PLHIV [1721]. Among these populations, a number of forms of stigma have been identified, including internalized, perceived, experienced, layered, and secondary stigmas [22]. Briefly, internalized stigma refers to a form of self-stigmatization whereby individuals accept negative judgments or attitudes applied to them [22]. Perceived or anticipated stigma is an awareness of devalued social status or expectation of discrimination based on a particular attribute [23]. Experienced or enacted stigma is the experience of a specific episode of discrimination against those with the stigmatized attribute or behaviour [24]. Secondary or courtesy stigma is stigma associated with those who have a connection with stigmatized individuals, such as their family or service providers [1, 25]. Layered or intersectional stigma [25] involves stigmas based on more than one attribute such as MSM living with HIV [25].

With growing recognition of the importance of stigma, there has been increasing interest and investment in stigma mitigation interventions [26]. Consequently, valid and reliable measures of stigma are needed to assess the impacts of these interventions and any changes in stigma over time [20]. Systematic reviews have examined measures of stigma affecting PLHIV [27]; less is known about measures of stigma affecting key populations whose sexual practices may put them at risk for HIV, including MSM and SW. An existing review on measuring attitudes towards homosexual men focused on stigma affecting a gay sexual orientation [28]. The systematic review presented here aims to summarize and synthesize studies that used quantitative, qualitative, or mixed methods to measure stigma affecting MSM and SW. Specifically, this review aims to systematically characterize how stigma associated with SW and with MSM is being measured and what validated and reliable stigma metrics exist for these key populations.

Methods

Search strategy

A scoping review of existing literature informed the development of this review’s search strategy and protocol (http://dx.doi.org/10.17504/protocols.io.ka6cshe). The search strategy used controlled vocabulary and subject headings, free text, and associated terms for both the stigma and key populations (SW and MSM) (Search strategies in S1 Text). Briefly, the Boolean operators “AND” and “OR” were used to combine the concepts. Cross-referencing concepts provided a broad, sensitive strategy to capture potentially relevant articles on SW- or MSM-associated stigma. Measurement-related terms were integrated in the abstract and full-text screening stages to identify relevant articles for inclusion, as including a measurement concept in the database search strategy created overly specific searches missing a priori determined key manuscripts. The base search was developed in the National Libraries of Medicine (PubMed) and adjusted according to other databases’ specifications. The following six databases were searched for peer-reviewed articles: PsycINFO, PubMed, EMBASE, CINAHL Plus, Global Health, and World Health Organization (WHO) Global Health Library Regional Indexes (AIM, LILACS, IMEMR, IMSEAR, and WPRIM).

Eligibility criteria

Inclusion criteria.

This review included primary research studies using quantitative, qualitative, and mixed methods data collection for the measurement of stigma associated with SW and/or MSM and published in English, French, or Spanish between January 1, 2004 and March 26, 2014. This timeframe of a decade was used to provide sufficient historical perspective on trends in stigma measurement. There were no restrictions on study design, duration or setting, country of study or publication, or on study population – including populations affected by stigma, perpetrators of stigma, students, healthcare workers – where stigma affecting SW or MSM was measured. Moreover, there were no restrictions on definitions, characteristics, identities or sexual practices of SW or MSM. For example, SW of any biological sex or gender identity, age, and race or ethnicity were included, and there were no limitations on definitions or types of sex work, nor on the duration or frequency of selling sex. MSM of any age, race or ethnicity were included, and there were no restrictions on type, duration, or frequency of same-sex sexual practices, including whether MSM had sex exclusively with other men, or also with women and/or transgender persons.

There were no limitations placed on the type, frequency, or duration of stigma associated with SW or MSM. The primary types of stigma of interest were decided through the scoping review and included internalized, perceived, and experienced stigmas, although studies including secondary/courtesy or other types of SW- or MSM-associated stigma were not excluded. Studies measuring MSM- or SW-associated stigma using pre-existing or new scales were included, regardless of whether validity or reliability were assessed, though levels of use of validated scales were noted.

Exclusion criteria.

Studies measuring stigma without any form of a scale, with a single question, and/or using binary/dichotomous variables not combined into a scale (e.g. “Do you feel stigmatized?”) were excluded from final data abstraction. Dissertations and theses not published in peer-reviewed journals were excluded. To effectively study stigma affecting transgender populations, different and specific search strategies are required. Here, we did not exclude studies that included transgender people in larger studies with cisgender MSM though noted these when included, but we did not include transgender-specific stigma studies.

Screening and abstraction

Independent reviewers were paired (e.g., reviewers 1 and 2; 1 and 3; 4 and 5) and each article was screened by two independent reviewers at the title and abstract (n = 6,470 entries) and full-text (n = 740 articles) review stages. Potentially relevant French and Spanish studies that had their titles and abstracts also translated into English were reviewed in the above manner and included in the above count. Otherwise, French and Spanish articles selected for full-text review – and those that subsequently met criteria for data abstraction – were completed by team members fluent in French or Spanish. Due to resource constraints, these few French and Spanish articles were completed by the single independent reviewer fluent in that language.

All English articles coded as potentially relevant by both reviewers were included for the next stage of the review process. If only one reviewer coded an article as potentially relevant during abstract screening, the review team included that entry for full-text review for increased sensitivity. After full-text review, discrepancies between reviewers regarding inclusion for data abstraction were resolved through discussions between the reviewers and another team member until consensus was reached.

Standardized forms were piloted and used for all screening phases and for data abstraction, per the search protocol. Data were abstracted by one reviewer for each included study using the developed standardized form, with a second reviewer independently examining 15% of articles and verifying their data abstraction. Independent dual abstraction of all included studies was not feasible due to resource constraints and the volume of included studies. The data abstraction form (S1 File) included information about study design and methods, study participants, target key population, and elements of stigma measurement, including scale, reliability, and validity. The form also included types of stigma (e.g. internalized, perceived, experienced); “N/A” was selected when the point of view was stigma perpetrators and the stigma types were not characterized.

Results

The initial search strategy identified 16,717 entries between six electronic databases, of which 5,134 duplicates were removed and 5,213 were excluded as non-peer reviewed publications or those published before 2004. Titles and abstracts of the remaining 6,370 entries were screened: 5,630 (88.4%) were excluded based on eligibility criteria and 740 (11.6%) papers were eligible for full text review. Of these 740 articles, 199 (26.9%) were excluded and 541 (73.1%) articles were included in this review for data abstraction (S2 Text). Percent agreement between reviewer pairs ranged from 83% to 87% for abstract screening, and from 84% to 90% for full-text review. For details on the screening process, see the flowchart in Fig 1.

thumbnail
Fig 1. Flowchart of screening process for inclusion of articles measuring stigma affecting men who have sex with men (MSM) and stigma affecting sex workers (SW), 2004-2014.

https://doi.org/10.1371/journal.pone.0188393.g001

Study characteristics

General characteristics of included studies are displayed in Table 1. Most articles (500/541; 92.4%) used quantitative methods, four (0.7%) used qualitative methods, and 37 (6.8%) used mixed methods. The majority of articles reported findings from North America: the United States (369/541; 68.2%) and Canada (19/541; 3.5%). Of the 369 articles from the United States, three (0.8%) measured SW stigma, one (0.3%) measured stigma for both SW and MSM, and the remaining 365 (98.9%) measured stigma for MSM populations. When grouped by WHO Regions [29], seven (1.3%) studies were from Sub-Saharan Africa (where HIV prevalence is highest) and measured MSM stigma. Seven (1.3%) articles reported findings from multiple regions. Location of data collection was unspecified in one (0.2%) article. Fig 2 provides the geographic distribution of included articles where data collection locations were specified, and Table 2 provides counts of each geographic location identified in included stigma measurement articles for SW only, SW and MSM, MSM only articles, as well as in papers identifying the inclusion of transgender persons.

thumbnail
Fig 2. Geographic distribution of all included articles measuring stigma toward men who have sex with men (MSM) and stigma toward sex workers (SW) by country, 2004-2014.

Not included in Fig 2 are studies that were Europe-wide [30, 31] and global without countries specified [32].

https://doi.org/10.1371/journal.pone.0188393.g002

thumbnail
Table 1. General characteristics of studies measuring stigma associated with men who have sex with men (MSM) and stigma associated with sex workers (SW) in articles from 2004-2014.

https://doi.org/10.1371/journal.pone.0188393.t001

thumbnail
Table 2. Distribution of identified geographic locations in included stigma measurement articles for sex workers (SW) only, SW and men who have sex with men (MSM), MSM only, and in papers including transgender persons, 2004-2014a.

https://doi.org/10.1371/journal.pone.0188393.t002

Regarding study samples in the 541 articles, 118 (21.8%) articles’ study populations were MSM only, 182 (33.6%) studies’ populations were MSM and another population (not including SWs), seven (1.3%) studies’ populations were SW only, and one (0.2%) study population was SW and another population (not including MSM). Three (0.6%) articles’ study populations included both MSM and SW. Thirty-four (6.3%) articles did not detail study sample composition, and 196 (36.2%) explicitly specified study samples other than MSM or SW (e.g. university students, health care workers, teachers, community members).

Stigma measure characteristics and metrics

Stigmatized populations addressed.

The majority (525/541; 97.0%) of studies measured stigma affecting gay men and other MSM. Of these 525 papers, 53 (10.1%) measured MSM-associated stigma for cisgender males only, 53 (10.1%) MSM articles specified including transgender individuals in their focus, 10 (1.9%) focused on transgender populations and did not include cisgender MSM, and 409 (77.9%) did not specify whether they restricted to cisgender MSM (Table 1). Among 13 (2.4%) articles assessing SW-associated stigma, nine (69.2%) focused on female SW and four (30.8%) did not specify SW genders. In addition to the above counts, three (0.6%) articles assessed stigma associated with both MSM and SW populations. Among these three studies, one (33.3%) article focused on stigma toward female SW, one (33.3%) focused on male SW, and one (33.3%) did not specify SW genders; none of these three articles specified whether individuals were cisgender and/or transgender.

Types of stigma.

Of the 541 studies, stigma type could be categorized for 281 (51.4%) articles. Of these 281, 102 (36.3%) articles measured two forms of stigma, and 26 (9.3%) had three stigma types. Of these 281 papers, 195 (69.4%) measured internalized stigma, 126 (44.8%) assessed experienced stigma, and 110 (39.1%) measured perceived stigma.

Among 525 articles measuring MSM stigma only, 273 articles (52%) could be categorized as one of the pre-determined types of stigma of interest (internalized, experienced/enacted, perceived/anticipated). Of these 273 articles, 193 (70.7%) measured internalized stigma, 125 (45.8%) measured experienced stigma, and 104 (38.1%) measured perceived stigma. Among 13 articles addressing SW stigma only, we were able to categorize seven (53.8%). Of those, five (71.4%) measured perceived stigma, one (14.3%) measured experienced stigma, and one (14.3%) measured internalized stigma. Among three articles addressing both MSM and SW stigmas, stigma type could be categorized for one (33.3%) paper, which measured both internalized and experienced stigma.

Stigma scales.

Items from several stigma scales were commonly used and adapted. Some studies employed multiple stigma scales or components of different scales. Regarding reliability, 369 (68.2%) articles provided Cronbach’s alpha for the stigma scales used, while 79 (14.6%) articles referred to reliability of scales used but did not provide Cronbach’s alpha. Of the remaining 93 (17.2%) articles, 13 (2.4%) only reported reliability for some but not all stigma measures used, and 80 (14.8%) did not reference reliability. Fewer articles reported validity of measures used (Fig 3). Validated stigma measures were reported for 224 (41.4%) articles, 28 (5.2%) papers reported validation for some but not all stigma scales used, and 289 (53.4%) papers did not report on validity or did not use validated stigma metrics.

thumbnail
Fig 3. Percentage validated scales used to measure stigma associated with sex workers (SW), SW and men who have sex with men (MSM), and MSM in articles from 2004-2014.

https://doi.org/10.1371/journal.pone.0188393.g003

Among 525 articles addressing MSM stigma only, authors most commonly used items from the Attitudes Toward Lesbians and Gay Men (ATLG) Scale [33], which was used in 128 (24.4%) articles. Originally developed in 1984, the ATLG has been revised several times (e.g. 1987 [34], 1988 [35], 1993 [36], 1994 [37], 1997 [38], 1998 [39], 2004 [40]). The Reactions to Homosexuality Scale [41] was used in 19 (3.6%) articles. The Modern Homonegativity Scale (MHS) [42] and Experiences of Homophobia [43, 44] measures were each used in 18 (3.4%) articles. These and additional scales commonly used to assess MSM stigma are outlined in Table 3. Three hundred and sixty (68.6%) reported a Cronbach’s alpha for the MSM stigma scales used, 13 (2.4%) referenced reliability for some but not all scales used, and 75 (14.3%) did not reference reliability of stigma measures employed. Regarding validation, 220 (41.9%) articles reported validation of included stigma scales, 28 (5.3%) articles reported validation for some but not all measures used, and 277 (52.8%) did not report on validity or did not use validated metrics.

thumbnail
Table 3. Most commonly used scales measuring stigma associated with men who have sex with men (MSM) in articles from 2004-2014.

https://doi.org/10.1371/journal.pone.0188393.t003

Among 13 articles addressing SW stigma only, no single stigma scale was used more than once (Table 4). Of the 13 scales measuring SW stigma, six (46.2%) were created for the study in which they are referenced: Sex Worker Stigma Index [52]; Perceived Stigma of Purchasing Sex [53]; Attitudes Towards and Beliefs About Sex Work [53]; Perceived Stigma [54]; Self-perceived Stigma [55]; and Attitudes Toward Prostitutes and Prostitution Scale [56]. Of the 13 articles measuring SW-associated stigma, seven (53.8%) reported Cronbach’s alpha for the stigma scales used, one (7.7%) referred to reliability without providing Cronbach’s alpha, and five (38.5%) did not reference reliability of stigma measures used. Four (30.8%) articles reported validation of included stigma scales, and nine (69.2%) papers did not report on validity or did not use validated stigma metrics.

thumbnail
Table 4. Scales used to measure stigma associated with sex workers (SW) in articles from 2004-2014a.

https://doi.org/10.1371/journal.pone.0188393.t004

Among the three articles addressing both MSM and SW stigma, no scale was used more than once. The Overall Stigmatization Scale for a Vulnerable Group (OSSVG) [64], Dimensional Stigmatization Scales [64], and the Stigma Consciousness Questionnaire [51] were each used once to measure both MSM- and SW-related stigmas. In one (33.3%) article, scales for MSM and SW-related stigmas were created using modified AIDS stigma scales [60]. Two (66.7%) articles reported Cronbach’s alpha for the stigma scales used, and one (33.3%) referenced reliability without providing Cronbach’s alpha. One (33.3%) article reported validation for included metrics and two (66.7%) did not report on validity or did not use validated scales.

Discussion

This systematic review provides a comprehensive overview of how stigma affecting MSM and SW is being measured, with several relevant findings to inform future studies. Notably, while MSM and SW have been studied and characterized as being disproportionately affected by key health conditions globally, studies measuring MSM- and SW-associated stigmas have been conducted predominantly in high-income countries, with far fewer occurring in low- and middle-income settings, including Latin America, Sub-Saharan Africa, the Middle East, and North Africa. Separately, there is far less research measuring stigma affecting SW. Although there are a significant number of studies about MSM, these have traditionally focused on stigma related to sexual orientation or identity rather than sexual practices – which may limit utility in other settings given the cultural specificity of sexual orientation. Finally, there remains limited usage of validated indicators of stigma affecting these populations, suggesting that the field would greatly benefit from increased rigour in the measurement of stigma.

Ultimately, only 3.0% (16/541) of included studies specifically measured SW-associated stigma. Even fewer used existing validated stigma metrics developed for SW, which allow health workers to compare differences in stigma magnitude between individuals and groups, changes over time within individuals or groups, or differential effects of stigma reduction interventions. Some studies measured SW stigma by adapting existing stigma scales not specific to SW, such as The Social Interaction Scale [63, 65], The Prejudice Evaluation Scale [63], and The Stigmatization Scale (Short Scale) [59]. However, this review found a few scales created specifically for measuring SW-associated stigma, such as the Sex Worker Stigma Index [52], the Attitudes Towards Prostitutes and Prostitution Scale [56], the Sexual Network Questionnaire [66], and the cross-sectional scales developed by Zhang et al. (2013) [54] and Hong et al. (2010) [55] – where all scales but the latter two were reported as having been validated. Several potentially relevant studies identified through the search strategy were excluded in screening stages. While these studies acknowledged that stigma exists and may have identified public health imperatives or implemented public health interventions to address stigma for SW or empower SW to overcome stigmatizing attitudes, they did not actually operationally measure SW stigma (e.g. [6769]). Moreover, some screened articles were excluded as they measured SW stigma in binary terms or with only one assessment question [7072]. Overly simplified or single indicators limit the extent to which stigma can be quantified in a nuanced way. For the purposes of this review, we only included studies that explicitly measured SW-related stigma to understand the specific health needs of SW. However, given the diversity among SW – including male, female, and transgender SW – there is value in measuring the intersectionality of different stigmas affecting these populations [69], and to better understanding the potentially negative, synergistic effects of layered stigmas on SW [73].

The majority of studies examining MSM stigma have included a focus on sexual orientation constructs and the use of anti-gay/anti-homosexuality attitude scales [35, 41, 45], as Table 3 highlights. While these studies provide important information, they may be less sensitive in the measurement of stigma affecting men who have sex with other men but who do not self-identify as gay or homosexual [74], particularly in countries with different local terms and identities that do not fit easily into the MSM paradigm. Fewer studies have focused on measuring stigma associated with same-sex practices [74, 75]. Yet in some of the most stigmatizing environments [7679], there is often independence of sexual orientation and sexual practices [80]. The criminalization of same-sex practices may also challenge stigma measurement and interventions due to potential danger and difficulty in undertaking these important endeavours [79], as has been noted recently in Uganda [81].

Around the globe, there may be different local terms, definitions and identities regarding which persons may self-identify as and/or be recognized under the umbrella term transgender. Results from this review indicated that there has been increased study of stigma and discrimination experienced by transgender persons. For example, the Transsexual Prejudice Scale [82] was used to examine interventions aiming to reduce transgender prejudice. A few transgender-specific stigma scales have been developed and validated, such as: the Transphobia Scale [83]; the Chinese Attitudes toward Transgenderism and Transgender Civil Rights Scale [84]; the cross-culturally validated [85] Genderism and Transphobia Scale [86]; and the Perceptions of the Averseness of Discrimination Scale (PADS) [87], which measures discrimination related to transgender status, and discrimination based on race/ethnicity. However, this review found some MSM studies tended to combine transgender women with MSM, or did not include subgroup analyses when both transgender women and cisgender MSM were included in the same study [8890]. In a study outside the scope of this review, Bazargan & Galvan [91] adapted general scaled questions measuring perceived discrimination to evaluate transgender-specific maltreatment, assessing it among transgender women. Metrics that independently and specifically measure gender-related stigma among transgender populations are crucial, as is examining stigma associated with transgender women and with transgender men separately, given differential experiences and impacts of stigma between transgender men and women [92]. While there are some studies evaluating gender-related stigma experienced by transgender women [69, 91, 93, 94], there is a need for additional research measuring the intersectionality of gender-related, sexual practice-related, and HIV-related stigmas to further inform interventions.

Beyond the gaps identified above related to these key populations, this review highlighted the opportunity for increased standardization across settings in measurement and methodologically sound validation. Given heightened interest in the well-being of key populations around the world, there has been translation – and some validation – of stigma scales in different linguistic contexts, including Chinese [95103], Spanish [85, 104110], Turkish [111115], Italian [106, 116118], and Hebrew [119, 120]. Although increased stigma measurement across settings is an advance, there has been limited psychometric assessment of stigma metrics in many settings. Overall, the majority of included articles did not use validated stigma metrics. And while validation can take many forms – including content validity, face validity, and criterion-related validity – it represents an important component of ensuring appropriate measurement of stigma [121].

Content validation assesses whether a measure includes all important dimensions of stigma [121], and was done in certain studies, including one developing the Sexual Prejudice Scale [122]. Some included studies reported face validation of stigma metrics, relying on experts or members of affected populations to assess whether a scale’s items appeared to measure the right stigma concepts [122, 123]. Criterion-related validity can take different forms, including concurrent or construct validity. Concurrent validity is the extent a developed stigma metric corresponds with other established measures of the same concept [121]. Few included studies had multiple MSM- or SW-related stigma metrics compared with each other in the same questionnaire, though this is a common practice in studies of other types of stigma [124]. An exception was a study assessing the concurrent validity of the Multiple Discrimination Scale (MDS) by reporting its correlation with other instruments (e.g. the Internalized Sexual Orientation Stigma Measures [125]). Two types of construct validity are convergent and discriminant validity. Convergent validity – the extent a metric correlates with other related variables in the same datasets [121, 126] – was found for a scale assessing attitudes toward gay rights, as it correlated with gender and religion [121, 126]. Discriminant validity is the extent a metric is independent of other conceptually distinct measures in the same dataset [121]. For example, one included study measuring attitudes towards gay men measured and controlled for social desirability bias [127]. Another reviewed study demonstrated discriminant validity of the Internalized Homophobia Scale, reporting its lack of correlation with other distinct concepts: positive affect and hostility attitudes [115]. Reliability assesses a metric’s consistency [121] using internal consistency and test-retest methods. Most included studies assessed internal consistency by calculating or reporting Cronbach’s alpha of stigma measures. A test-retest approach to determine reliability (based on measuring the same concept twice [121]) was used in few studies: one demonstrated test-retest reliability of the Multiple Discrimination Scale through monthly administrations of the scale [125].

Some limitations of this review must be acknowledged, as well as the review’s strengths in synthesizing existing literature. The large volume of information reviewed and comprehensiveness of this review provided wide scope, but limited the ability to delve into specific details on particular aspects of stigma scale measurement. With the large volume of included articles, results were not dually abstracted for each study, although quality control was completed on 15% of extracted articles via independent review by a second reviewer. Any questions about a particular aspect of an article’s data abstraction were reviewed and discussed with other co-authors to maximize consensus and minimize subjectivity in the review process. Outside the comprehensive review of the six databases, no additional searches for unpublished or non-peer-reviewed sources were planned or undertaken, and hand searching of all 541 included articles’ reference lists was not conducted. However, including quantitative, qualitative and mixed methods research from six electronic databases and articles published in three widely-spoken languages helped minimize geographic, language, and publication bias, and represent strengths of this review. This large-scale review scoped and systematically characterized how and where SW- and MSM-associated stigma is being measured. Given this focus and the volume of included studies, critical appraisal was not done for each individual study included in this review. Additionally, while the methods used for this review were consistent with PRISMA guidelines, the protocol not was able to be registered in PROSPERO as the data collection had initiated before PROSPERO had emerged as the standard database in which to register systematic reviews. Finally, this review focused on cisgender MSM and did not appropriately include all of the terms for transgender men or women. If a study was focused on MSM without differentiating cisgender MSM from transgender women, then it was not excluded. A limitation of this review is that it did not adequately study approaches to measure transgender-related stigma which represents a key research question moving forward, especially given the intersectionality of sexual orientation, sexual practice, and gender-related stigma.

Conclusion

The improved measurement of stigma has great potential in guiding effective responses to a variety of health conditions disproportionately affecting key populations. And while these data show significant measurement of MSM-related stigma, this work has primarily been completed in high-income settings. Moreover, where completed, there remains inconsistent use of validated stigma metrics. Moving forward necessitates improved measurement of stigma affecting SW as well as transgender persons, and also increased work for all populations – particularly across Sub-Saharan Africa and the Middle East. While not the focus here, there are limited data on stigma affecting people who use drugs, and these populations were not included in this review. The data in this review suggests the opportunity for the use of validated scales – or more efforts to validate scales – when measuring MSM- or SW-associated stigmas in new settings. Though contexts differ, key populations exist in every society around the world. There continue to be epidemiologic and interventional efforts to comprehensively characterize the specific HIV prevention, treatment, and care needs of these populations. The effective integration of stigma metrics into these studies and services will provide the opportunity to characterize the ideal content of biomedical and behavioural approaches to decrease proximal HIV acquisition and transmission risks, as well as optimal implementation strategies for mitigating the barriers to uptake of those services among those most in need.

Acknowledgments

The authors wish to thank numerous individuals who contributed to this review, including: Carrie Lyons for assistance with Spanish language articles; Shianne Busch, Daniel Choi, Muskaan Gurnani, Harneel Kaur, and Ryan Max for assistance with data abstraction/analytic elements; Ivy Garner for de-duplication of articles; and Amanda Bowes for contributions to concept formulation.

References

  1. 1. Goffman E. Stigma; notes on the management of spoiled identity. Englewood Cliffs, N.J.,: Prentice-Hall; 1963. 147 p. p.
  2. 2. Crocker J, Major B, Steele C. Social Stigma. In: Fiske S, Gilbert D, Lindzey G, editors. Handbook of Social Psychology. 2. Boston, MA: McGraw-Hill; 1998. p. 504–53.
  3. 3. Corrigan PW, Larson JE, Rusch N. Self-stigma and the "why try" effect: impact on life goals and evidence-based practices. World Psychiatry. 2009;8(2):75–81. pmid:19516923
  4. 4. Jamison KR. The many stigmas of mental illness. Lancet. 2006;367(9509):533–4. pmid:16473132
  5. 5. Link BG, Phelan JC. Conceptualizing stigma. Annu Rev Sociol. 2001;27:363–85.
  6. 6. Rao D, Feldman BJ, Fredericksen RJ, Crane PK, Simoni JM, Kitahata MM, et al. A structural equation model of HIV-related stigma, depressive symptoms, and medication adherence. AIDS and behavior. 2012;16(3):711–6. pmid:21380495
  7. 7. Rao D, Chen WT, Pearson CR, Simoni JM, Fredriksen-Goldsen K, Nelson K, et al. Social support mediates the relationship between HIV stigma and depression/quality of life among people living with HIV in Beijing, China. International journal of STD & AIDS. 2012;23(7):481–4.
  8. 8. Hatzenbuehler ML, O'Cleirigh C, Mayer KH, Mimiaga MJ, Safren SA. Prospective associations between HIV-related stigma, transmission risk behaviors, and adverse mental health outcomes in men who have sex with men. Annals of behavioral medicine: a publication of the Society of Behavioral Medicine. 2011;42(2):227–34.
  9. 9. Hua J, Emrick CB, Golin CE, Liu K, Pan J, Wang M, et al. HIV and stigma in Liuzhou, China. AIDS and behavior. 2014;18 Suppl 2:S203–11.
  10. 10. Li X, Huang L, Wang H, Fennie KP, He G, Williams AB. Stigma mediates the relationship between self-efficacy, medication adherence, and quality of life among people living with HIV/AIDS in China. AIDS patient care and STDs. 2011;25(11):665–71. pmid:22023316
  11. 11. Golub SA, Tomassilli JC, Parsons JT. Partner serostatus and disclosure stigma: implications for physical and mental health outcomes among HIV-positive adults. AIDS and behavior. 2009;13(6):1233–40. pmid:18843532
  12. 12. Peitzmeier SM, Grosso A, Bowes A, Ceesay N, Baral SD. Associations of stigma with negative health outcomes for people living with HIV in the Gambia: implications for key populations. J Acquir Immune Defic Syndr. 2015;68 Suppl 2:S146–53.
  13. 13. Wolitski RJ, Pals SL, Kidder DP, Courtenay-Quirk C, Holtgrave DR. The effects of HIV stigma on health, disclosure of HIV status, and risk behavior of homeless and unstably housed persons living with HIV. AIDS and behavior. 2009;13(6):1222–32. pmid:18770023
  14. 14. Sweeney SM, Vanable PA. The Association of HIV-Related Stigma to HIV Medication Adherence: A Systematic Review and Synthesis of the Literature. AIDS and behavior. 2016;20(1):29–50. pmid:26303196
  15. 15. Hatzenbuehler ML, Phelan JC, Link BG. Stigma as a fundamental cause of population health inequalities. American journal of public health. 2013;103(5):813–21. pmid:23488505
  16. 16. Stagnor C, Crandall C. Threat and the social construction of stigma. In: Heatherton T, Kleck R, Hebl M, Hull J, editors. The social psychology of stigma. New York: The Guilford Press; 2000.
  17. 17. Brown L, Macintyre K, Trujillo L. Interventions to reduce HIV/AIDS stigma: what have we learned? AIDS education and prevention: official publication of the International Society for AIDS Education. 2003;15(1):49–69.
  18. 18. Earnshaw VA, Chaudoir SR. From conceptualizing to measuring HIV stigma: a review of HIV stigma mechanism measures. AIDS and behavior. 2009;13(6):1160–77. pmid:19636699
  19. 19. Reidpath DD, Chan KY. A method for the quantitative analysis of the layering of HIV-related stigma. AIDS care. 2005;17(4):425–32. pmid:16036227
  20. 20. Stangl AL, Lloyd JK, Brady LM, Holland CE, Baral S. A systematic review of interventions to reduce HIV-related stigma and discrimination from 2002 to 2013: how far have we come? Journal of the International AIDS Society. 2013;16(3 Suppl 2):18734.
  21. 21. van der Elst EM, Smith AD, Gichuru E, Wahome E, Musyoki H, Muraguri N, et al. Men who have sex with men sensitivity training reduces homoprejudice and increases knowledge among Kenyan healthcare providers in coastal Kenya. Journal of the International AIDS Society. 2013;16 Suppl 3:18748.
  22. 22. Jain A, Nyblade L. Scaling Up Policies, Interventions, and Measurement for Stigma-Free HIV Prevention, Care, and Treatment Services. Washington: Futures Group, Health Policy Project, 2012.
  23. 23. Brown R. Perceived stigma, medical social support and quality of life among people living with HIV/AIDS in Hunan, China. Applied Nursing Research. 2015;28(2):169–74. pmid:25315140
  24. 24. Lekas HM, Siegel K, Leider J. Felt and enacted stigma among HIV/HCV-coinfected adults: the impact of stigma layering. Qual Health Res. 2011;21(9):1205–19. pmid:21498828
  25. 25. Smit PJ, Brady M, Carter M, Fernandes R, Lamore L, Meulbroek M, et al. HIV-related stigma within communities of gay men: a literature review. AIDS care. 2012;24(4):405–12. pmid:22117138
  26. 26. Auerbach JD, Parkhurst JO, Caceres CF. Addressing social drivers of HIV/AIDS for the long-term response: conceptual and methodological considerations. Glob Public Health. 2011;6 Suppl 3:S293–309.
  27. 27. Logie C, Gadalla TM. Meta-analysis of health and demographic correlates of stigma towards people living with HIV. AIDS care. 2009;21(6):742–53. pmid:19806490
  28. 28. Grey JA RB, Coleman E, Bockting WO. A systematic review of instruments that measure attitudes toward homosexual men. Journal of sex research. 2013;50(3-4):329–52. pmid:23480076
  29. 29. World Health Organization. Definition of region groupings: World Health Organization,; 2016 [Available from: http://www.who.int/healthinfo/global_burden_disease/definition_regions/en/.
  30. 30. Ross MW, Berg RC, Schmidt AJ, Hospers HJ, Breveglieri M, Furegato M, et al. Internalised homonegativity predicts HIV-associated risk behavior in European men who have sex with men in a 38-country cross-sectional study: some public health implications of homophobia. BMJ open. 2013;3(2).
  31. 31. Berg RC, Ross MW, Weatherburn P, Schmidt AJ. Structural and environmental factors are associated with internalised homonegativity in men who have sex with men: findings from the European MSM Internet Survey (EMIS) in 38 countries. Social science & medicine (1982). 2013;78:61–9.
  32. 32. Oth VT, Lindner NM, Nosek BA. Do unto others: Effects of priming the golden rule on Buddhists and Christians attitudes toward gay people. Journal for the Scientific Study of Religion. 2010;49(3):494–506.
  33. 33. Herek GM. Attitudes toward lesbians and gay men: A factor analytic study. Journal of homosexuality. 1984;10(1/2):39–51.
  34. 34. Herek GM. Religion and prejudice: A comparison of racial and sexual attitudes. Personality and Social Psychology Bulletin. 1987;13:56–65.
  35. 35. Herek GM. Heterosexuals' attitudes toward lesbians and gay men: Correlates and gender differences. Journal of sex research. 1988;25:451–77.
  36. 36. Herek GM, Glunt EK. Interpersonal contact and heterosexuals' attitudes toward gay men: Results from a national survey. The Journal of Sex Research. 1993;30(3):239–44.
  37. 37. Herek GM. Assessing heterosexuals' attitudes toward lesbians and gay men: A review of empirical research with the ATLG scale. In: Greene GMHE B., editor. Lesbian and gay psychology: Theory, research, and clinical applications. Thousand Oaks, CA: Sage Publications; 1994. p. 206–28.
  38. 38. Herek GM. The HIV epidemic and public attitudes toward lesbians and gay men. In: Levine PN M.P., & Gagnon J. (Eds.), editor. In changing times: The impact of the HIV epidemic on the lesbian and gay community. Chicago, IL: University of Chicago Press; 1997.
  39. 39. Herek GM. The Attitudes Toward Lesbians and Gay Men (ATLG) scale. In: Davis WHY C.M., Bauserman R., Schreer G., & Davis S.L. (Eds.), editor. Sexuality-related measures: A compendium. Thousand Oaks, CA: Sage Publications; 1998.
  40. 40. Herek GM. The Attitudes Toward Lesbians and Gay Men Scale Davis, CA: Department of Psychology, University of California, Davis; 2004 [Available from: http://psychology.ucdavis.edu/rainbow/html/atlg.html.
  41. 41. Ross MW RB. Measurement and correlates of internalized homophobia: a factor analytic study. J Clin Psychol. 1996;52(1):15–21. pmid:8682906
  42. 42. Morrison MA, Morrison TG. Development and validation of a scale measuring modern prejudice toward gay men and lesbian women. Journal of homosexuality. 2002;43(2):15–37. pmid:12739696
  43. 43. Diaz RM, Ayala G, Bein E, Henne J, Marin BV. The impact of homophobia, poverty, and racism on the mental health of gay and bisexual Latino men: findings from 3 US cities. American journal of public health. 2001;91(6):927. pmid:11392936
  44. 44. Diaz RM, Ayala G, Bein E. Sexual risk as an outcome of social oppression: data from a probability sample of Latino gay men in three U.S. cities. Cultural diversity & ethnic minority psychology. 2004;10(3):255–67.
  45. 45. Shidlo A. Internalized homophobia: Conceptual and empirical issues in measurement. In: Greene BEH, Gregory M. (Ed), editor. Lesbian and gay psychology: Theory, research, and clinical applications Psychological perspectives on lesbian and gay issues. 1. Thousand Oaks, CA, US: Sage Publications, Inc; 1994. p. 176–205.
  46. 46. Mayfield W. The development of an Internalized Homonegativity Inventory for gay men. Journal of homosexuality. 2001;41(2):53–76. pmid:11482428
  47. 47. Hudson WW, Ricketts WA. A strategy for the measurement of homophobia. Journal of homosexuality. 1980;5(4):357–72. pmid:7204951
  48. 48. Martin J, Dean L. Ego-Dystonic Homosexuality Scale. USA: Columbia University; 1987.
  49. 49. Martin J, Dean L. Mental health effects of Aids on at-risk homosexual men. 1992.
  50. 50. Nungesser LG. Homosexual acts, actors, and identities. New York, NY: Praeger; 1983. xix, 215 p. p.
  51. 51. Pinel EC. Stigma consciousness: the psychological legacy of social stereotypes. Journal of personality and social psychology. 1999;76(1):114–28. pmid:9972557
  52. 52. Liu SH, Srikrishnan AK, Zelaya CE, Solomon S, Celentano DD, Sherman SG. Measuring perceived stigma in female sex workers in Chennai, India. AIDS care. 2011;23(5):619–27. pmid:21293991
  53. 53. Pitpitan EV, Strathdee SA, Semple SJ, Wagner KD, Chavarin CV, Earnshaw VA, et al. Perceived Stigma of Purchasing Sex Among Latino and Non-Latino Male Clients of Female Sex Workers in Tijuana, Mexico. Journal of immigrant and minority health / Center for Minority Public Health. 2013.
  54. 54. Zhang C, Li X, Hong Y, Su S, Zhou Y. Relationship between female sex workers and gatekeeper: The impact on female sex worker's mental health in China. Psychology, health & medicine. 2013.
  55. 55. Hong Y, Fang X, Li X, Liu Y, Li M, Tai-Seale T. Self-perceived stigma, depressive symptoms, and suicidal behaviors among female sex workers in China. Journal of transcultural nursing: official journal of the Transcultural Nursing Society / Transcultural Nursing Society. 2010;21(1):29–34.
  56. 56. Levin L, Peled E. The Attitudes Toward Prostitutes and Prostitution Scale: A New Tool for Measuring Public Attitudes Toward Prostitutes and Prostitution. Research on Social Work Practice. 2011;21(5):582–93.
  57. 57. Basow SA, Campanile F. Attitudes toward Prostitution as a Function of Attitudes toward Feminism in College-Students - an Exploratory-Study. Psychol Women Quart. 1990;14(1):135–41.
  58. 58. Genberg BL, Hlavka Z, Konda KA, Maman S, Chariyalertsak S, Chingono A, et al. A comparison of HIV/AIDS-related stigma in four countries: negative attitudes and perceived acts of discrimination towards people living with HIV/AIDS. Social science & medicine (1982). 2009;68(12):2279–87.
  59. 59. Harvey RD. Individual differences in the phenomenological impact of social stigma. J Soc Psychol. 2001;141(2):174–89. pmid:11372564
  60. 60. Held SL. The effects of an AIDS education program on the knowledge and attitudes of a physical therapy class. Phys Ther. 1993;73(3):156–64. pmid:8438003
  61. 61. Jehu D, Gazan M, Klassen C. Beyond sexual abuse: therapy with women who were childhood victims. Chichester; New York: Wiley; 1988. xvi, 352 p. p.
  62. 62. Kamise Y. Occupational stigma and coping strategies of women engaged in the commercial sex industry: A study on the perception of 'kyaba-cula hostesses' in Japan. Sex Roles. 2013;69(1-2):42–57.
  63. 63. Kelly JA, St Lawrence JS, Smith S Jr., Hood HV, Cook DJ. Stigmatization of AIDS patients by physicians. American journal of public health. 1987;77(7):789–91. pmid:3592030
  64. 64. Lau JT, Choi KC, Tsui HY, Su X. Associations between stigmatization toward HIV-related vulnerable groups and similar attitudes toward people living with HIV/AIDS: branches of the same tree? AIDS care. 2007;19(10):1230–40. pmid:18071967
  65. 65. Kelly JA, St Lawrence JS, Smith S Jr., Hood HV, Cook DJ. Medical students' attitudes toward AIDS and homosexual patients. J Med Educ. 1987;62(7):549–56. pmid:3599050
  66. 66. Klovdahl A. Social networks and the spread of infectious diseases: the AIDS example. Social Science and Medicine. 1985;21(11):1203–16. pmid:3006260
  67. 67. Mirzazadeh A, Nedjat S, Navadeh S, Haghdoost A, Mansournia M-A, McFarland W, et al. Hiv and related risk behaviors among female sex workers in iran: Bias-adjusted estimates from the 2010 national bio-behavoral survey. AIDS and behavior. 2013.
  68. 68. Kerrigan D, Telles P, Torres H, Overs C, Castle C. Community development and HIV/STI-related vulnerability among female sex workers in Rio de Janeiro, Brazil. Health education research. 2008;23(1):137–45. pmid:17363361
  69. 69. Nemoto T, Bodeker B, Iwamoto M. Social support, exposure to violence and transphobia, and correlates of depression among male-to-female transgender women with a history of sex work. American journal of public health. 2011;101(10):1980–8. pmid:21493940
  70. 70. Groves J, Newton DC, Chen MY, Hocking J, Bradshaw CS, Fairley CK. Sex workers working within a legalised industry: their side of the story. Sexually transmitted infections. 2008;84(5):393–4. pmid:18550694
  71. 71. Lazarus L, Deering KN, Nabess R, Gibson K, Tyndall MW, Shannon K. Occupational stigma as a primary barrier to health care for street-based sex workers in Canada. Culture, health & sexuality. 2012;14(2):139–50.
  72. 72. Norman LR, Carr R, Jimenez J. Sexual stigma and sympathy: attitudes toward persons living with HIV in Jamaica. Culture, health & sexuality. 2006;8(5):423–33.
  73. 73. Chan KY, Stoove MA, Sringernyuang L, Reidpath DD. Stigmatization of AIDS patients: disentangling Thai nursing students' attitudes towards HIV/AIDS, drug use, and commercial sex. AIDS and behavior. 2008;12(1):146–57. pmid:17364148
  74. 74. Heath J, Goggin K. Attitudes towards male homosexuality, bisexuality, and the down low lifestyle: Demographic differences and HIV implications. Journal of Bisexuality. 2009;9(1):17–31.
  75. 75. Logie CH, Newman PA, Chakrapani V, Shunmugam M. Adapting the minority stress model: associations between gender non-conformity stigma, HIV-related stigma and depression among men who have sex with men in South India. Social science & medicine (1982). 2012;74(8):1261–8.
  76. 76. Ahrold TK, Meston CM. Ethnic differences in sexual attitudes of U.S. college students: Gender, acculturation, and religiosity factors. Archives of sexual behavior. 2010;39(1):190–202. pmid:18839302
  77. 77. Barbosa P, Torres H, Silva MA, Khan N. Agapé Christian reconciliation conversations: Exploring the intersections of culture, religiousness, and homosexual identity in Latino and European Americans. Journal of homosexuality. 2010;57(1):98–116. pmid:20069496
  78. 78. Barnes DM, Meyer IH. Religious affiliation, internalized homophobia, and mental health in lesbians, gay men, and bisexuals. The American journal of orthopsychiatry. 2012;82(4):505–15. pmid:23039348
  79. 79. Carroll A, Itaborahy LP, International Lesbian G, Bisexual, Trans and Intersex Association. State Sponsored Homophobia 2015: A world survey of laws: criminalisation, protection and recognition of same-sex love. Geneva: International Lesbian, Gay, Bisexual, Trans and Intersex Association (ILGA), 2015 May 2015. Report No.
  80. 80. Trapence G CC, Avrett S, Carr R, Sanchez H, Ayala G, Diouf D, Beyrer C, Baral SD. From personal survival to public health: community leadership by men who have sex with men in the response to HIV. Lancet. 2012;380(9839):400–10. pmid:22819662
  81. 81. Ross MW, Kajubi P, Mandel JS, McFarland W, Raymond HF. Internalized homonegativity/homophobia is associated with HIV-risk behaviours among Ugandan gay and bisexual men. International journal of STD & AIDS. 2013;24(5):409–13.
  82. 82. Case KA, Stewart B. Intervention Effectiveness in Reducing Prejudice Against Transsexuals. Journal of LGBT Youth. 2013;10(1/2):140–58.
  83. 83. Nagoshi JL, Adams KA, Terrell HK, Hill ED, Brzuzy S, Nagoshi CT. Gender differences in correlates of homophobia and transphobia. Sex Roles. 2008;59(7-8):521–31.
  84. 84. King ME, Winter S, Webster B. Contact reduces transprejudice: a study on attitudes towards transgenderism and transgender civil rights in Hong Kong. International Journal of Sexual Health. 2009;21(1):17–34.
  85. 85. Carrera-Fernández MV, Lameiras-Fernández M, Rodriguez-Castro Y, Vallejo-Medina P. Spanish Adolescents' Attitudes toward Transpeople: Proposal and Validation of a Short Form of the Genderism and Transphobia Scale. The Journal of Sex Research. 2014;51(6):654–66. pmid:23767992
  86. 86. Hill DB, Willoughby BLB. The Development and Validation of the Genderism and Transphobia Scale. Sex Roles. 2005;53(7/8):531–44.
  87. 87. Erich S, Tittsworth J, Meier SLC, Lerman T. Transsexuals of Color: Perceptions of discrimination based on transsexual status and race/ethnicity status. Journal of GLBT Family Studies. 2010;6(3):294–314.
  88. 88. Dowshen N, Binns HJ, Garofalo R. Experiences of HIV-related stigma among young men who have sex with men. AIDS patient care and STDs. 2009;23(5):371–6. pmid:19320600
  89. 89. Kuhns LM, Vazquez R, Ramirez-Valles J. Researching special populations: retention of Latino gay and bisexual men and transgender persons in longitudinal health research. Health education research. 2008;23(5):814–25. pmid:17974545
  90. 90. Ramirez-Valles J, Molina Y, Dirkes J. Stigma towards PLWHA: the role of internalized homosexual stigma in Latino gay/bisexual male and transgender communities. AIDS education and prevention: official publication of the International Society for AIDS Education. 2013;25(3):179–89.
  91. 91. Bazargan M, Galvan F. Perceived discrimination and depression among low-income Latina male-to-female transgender women. BMC public health. 2012;12:663. pmid:22894701
  92. 92. Winter S, Webster B, Cheung PKE. Measuring Hong Kong undergraduate students' attitudes towards transpeople. Sex Roles. 2008;59(9-10):670–83.
  93. 93. Winter S, Chalungsooth P, Teh YK, Rojanalert N, Maneerat K, Wong YW, et al. Transpeople, transprejudice and pathologization: a seven-country factor analytic study. International Journal of Sexual Health. 2009;21(2):96–118.
  94. 94. Golub SA, Walker JJ, Longmire-Avital B, Bimbi DS, Parsons JT. The role of religiosity, social support, and stress-related growth in protecting against HIV risk among transgender women. Journal of health psychology. 2010;15(8):1135–44. pmid:20522502
  95. 95. Yu Y, Xiao S, Xiang Y. Application and testing the reliability and validity of a modified version of Herek's Attitudes Toward Lesbians and Gay Men Scale in China. Journal of homosexuality. 2011;58(2):263–74. pmid:21294029
  96. 96. Neilands TB, Steward WT, Choi KH. Assessment of stigma towards homosexuality in China: a study of men who have sex with men. Archives of sexual behavior. 2008;37(5):838–44. pmid:18274889
  97. 97. Liu H, Feng T, Rhodes AG, Liu H. Assessment of the Chinese version of HIV and homosexuality related stigma scales. Sexually transmitted infections. 2009;85(1):65–9. pmid:18790858
  98. 98. Pyun T, Santos GM, Arreola S, Do T, Hebert P, Beck J, et al. Internalized Homophobia and Reduced HIV Testing Among Men Who Have Sex With Men in China. Asia-Pacific journal of public health / Asia-Pacific Academic Consortium for Public Health. 2014.
  99. 99. Hu X, Wang Y. LGB identity among young Chinese: The influence of traditional culture. Journal of homosexuality. 2013;60(5):667–84. pmid:23593953
  100. 100. Wu J, Kwok DK. Psychometric properties of Attitudes Towards Lesbians and Gay Men Scale with Chinese university students. Psychological Reports. 2012;110(2):521–6. pmid:22662406
  101. 101. Tu J, Lee TT. The Effects of Media Usage and Interpersonal Contacts on the Stereotyping of Lesbians and Gay Men in China. Journal of homosexuality. 2013.
  102. 102. Chan KY, Reidpath DD. Stigmatization of patients with AIDS: understanding the interrelationships between Thai nurses' attitudes toward HIV/AIDS, drug use, and commercial sex. AIDS patient care and STDs. 2007;21(10):763–75. pmid:17949275
  103. 103. Kwok DK, Wu J, Shardlow SM. Attitudes toward lesbians and gay men among Hong Kong Chinese social work students. Journal of Social Work Education. 2013;49(2):337–52.
  104. 104. Cárdenas M, Barrientos JE. The attitudes toward lesbians and gay men scale (ATLG): adaptation and testing the reliability and validity in Chile. Journal of sex research. 2008;45(2):140–9. pmid:18569535
  105. 105. Cárdenas M, Barrientos J, Gómez F, Frías-Navarro D. Attitudes Toward Gay Men and Lesbians and Their Relationship with Gender Role Beliefs in a Sample of Chilean University Students. International Journal of Sexual Health. 2012;24(3):226–36.
  106. 106. Baiocco R, Ioverno S, Cerutti R, Santamaria F, Fontanesi L, Lingiardi V, et al. Suicidal ideation in Spanish and Italian lesbian and gay young adults: The role of internalized sexual stigma. Psicothema. 2014;26(4):490–6. pmid:25340896
  107. 107. Barrientos J, Cárdenas M, Gómez F, Frías-Navarro D. Assessing the dimensionality of Beliefs About Children's Adjustment in Same-Sex Families Scale (BCASSFS) in Chile. Sexuality Research & Social Policy: A Journal of the NSRC. 2013;10(1):43–51.
  108. 108. Delgado JB, Castro MC. Construction and validation of a subjective scale of stigma and discrimination (sisd) for the gay men and transgender women population in chile. Sexuality Research & Social Policy: A Journal of the NSRC. 2014.
  109. 109. Nierman AJ, Thompson SC, Bryan A, Mahaffey AL. Gender role beliefs and attitudes toward lesbians and gay men in Chile and the U.S. Sex Roles. 2007;57(1-2):61–7.
  110. 110. Perez-Testor C, Behar J, Davins M, Conde Sala JL, Castillo JA, Salamero M, et al. Teachers' attitudes and beliefs about homosexuality. The Spanish journal of psychology. 2010;13(1):138–55. pmid:20480684
  111. 111. Saraç L. The relationships between homophobic attitudes and religiosity among Turkish physical education teacher majors. Physical Education and Sport Pedagogy. 2012;17(3):277–87.
  112. 112. Saraç L. Attitudes of future physical education teachers in Turkey toward lesbians and gay men. Psychological Reports. 2012;111(3):765–75. pmid:23402046
  113. 113. Gelbal S, Duyan V. Attitudes of university students toward lesbians and gay men in Turkey. Sex Roles. 2006;55(7-8):573–9.
  114. 114. Gomleksiz M, Poyrazli S, Vural RA. Discriminatory attitudes: A scale development in Turkish. Psychological Reports. 2008;103(3):921–30. pmid:19320229
  115. 115. Gencoz T, Yuksel M. Psychometric properties of the Turkish version of the internalized homophobia scale. Archives of sexual behavior. 2006;35(5):597–602. pmid:17031586
  116. 116. Prati G, Pietrantoni L, D'Augelli AR. Aspects of homophobia in Italian high schools: Students' attitudes and perceptions of school climate. Journal of Applied Social Psychology. 2011;41(11):2600–20.
  117. 117. Baiocco R, D'Alessio M, Laghi F. Binge drinking among gay, and lesbian youths: The role of internalized sexual stigma, self-disclosure, and individuals' sense of connectedness to the gay community. Addictive behaviors. 2010;35(10):896–9. pmid:20584573
  118. 118. Baiocco R, Nardelli N, Pezzuti L, Lingiardi V. Attitudes of italian heterosexual older adults towards lesbian and gay parenting. Sexuality Research & Social Policy: A Journal of the NSRC. 2013.
  119. 119. Shilo G, Savaya R. Mental health of lesbian, gay, and bisexual youth and young adults: Differential effects of age, gender, religiosity, and sexual orientation. Journal of Research on Adolescence. 2012;22(2):310–25.
  120. 120. Kulik L. Transmission of Attitudes Regarding Family Life From Parents to Adolescents in Israel. Families in Society. 2004;85(3):345–53.
  121. 121. Remler D, Van Ryzin G. Research methods in practice: strategies for description and causation. Thousand Oaks, California: SAGE Publications; 2011.
  122. 122. Chonody JM. Measuring sexual prejudice against gay men and lesbian women: development of the Sexual Prejudice Scale (SPS). Journal of homosexuality. 2013;60(6):895–926. pmid:23688314
  123. 123. Currie MR, Cunningham EG, Findlay BM. The Short Internalized Homonegativity Scale: examination of the factorial structure of a new measure of internalized homophobia. Educational & Psychological Measurement. 2004;64(6):1053–67.
  124. 124. Vogel D, Wade N, Ascheman P. Measuring perceptions of stigmatization by others for seeking psychological help: Reliability and validity of a new stigma scale with college students. Journal of counseling psychology. 2009;56(2):301–8.
  125. 125. Bogart LM, Landrine H, Galvan FH, Wagner GJ, Klein DJ. Perceived discrimination and physical health among HIV-positive Black and Latino men who have sex with men. AIDS and behavior. 2013;17(4):1431–41. pmid:23297084
  126. 126. Hooghe M, Claes E, Harell A, Quintelier E, Dejaeghere Y. Anti-gay sentiment among adolescents in Belgium and Canada: a comparative investigation into the role of gender and religion. Journal of homosexuality. 2010;57(3):384–400. pmid:20391000
  127. 127. Einbinder SD, Fiechter S, Sheridan DA, Miller DL. Social work educators’ attitudes toward gay men and lesbians: A national assessment. Journal of Gay & Lesbian Social Services: The Quarterly Journal of Community & Clinical Practice. 2012;24(2):173–200.