Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

The stigma associated with cutaneous leishmaniasis (CL) and mucocutaneous leishmaniasis (MCL): A protocol for a systematic review

  • Hasara Nuwangi,

    Roles Conceptualization, Methodology, Writing – original draft, Writing – review & editing

    Affiliation Department of Community Medicine, Faculty of Medicine and Allied Sciences, Rajarata University of Sri Lanka, Anuradhapura, Sri Lanka

  • Thilini Chanchala Agampodi,

    Roles Conceptualization, Methodology, Supervision, Writing – review & editing

    Affiliation Department of Community Medicine, Faculty of Medicine and Allied Sciences, Rajarata University of Sri Lanka, Anuradhapura, Sri Lanka

  • Helen Philippa Price,

    Roles Conceptualization, Funding acquisition, Methodology, Supervision, Writing – review & editing

    Affiliation School of Life Sciences, Keele University, Newcastle-under-Lyme, Staffordshire, United Kingdom

  • Thomas Shepherd,

    Roles Conceptualization, Methodology, Supervision

    Affiliation School of Medicine, Keele University, Newcastle-under-Lyme, Staffordshire, United Kingdom

  • Kosala Gayan Weerakoon ,

    Contributed equally to this work with: Kosala Gayan Weerakoon, Suneth Buddhika Agampodi

    Roles Conceptualization, Methodology, Supervision, Writing – review & editing

    Affiliation Department of Parasitology, Faculty of Medicine and Allied Sciences, Rajarata University of Sri Lanka, Anuradhapura, Sri Lanka

  • Suneth Buddhika Agampodi

    Contributed equally to this work with: Kosala Gayan Weerakoon, Suneth Buddhika Agampodi

    Roles Conceptualization, Funding acquisition, Methodology, Project administration, Supervision, Writing – review & editing

    sunethagampodi@yahoo.com

    Affiliations Department of Internal Medicine, Section of Infectious Diseases, Yale University School of Medicine, New Haven, Connecticut, United States of America, International Vaccine Institute, Seoul, Republic of Korea

Abstract

Leishmaniasis is a neglected tropical disease with three main clinical types; cutaneous leishmaniasis (CL), mucocutaneous leishmaniasis (MCL), and visceral leishmaniasis (VL). CL and MCL are considered to be highly stigmatizing due to potentially disfiguring skin pathology. CL and MCL-associated stigma are reported across the world in different contexts assimilating different definitions and interpretations. Stigma affects people with CL, particularly in terms of quality of life, accessibility to treatment, and psycho-social well-being. However, evidence on CL- and MCL-associated stigma is dispersed and yet to be synthesized. This systematic review describes the types, measurements, and implications of the stigma associated with CL and MCL and identifies any preventive strategies/interventions adopted to address the condition. This study was developed according to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols (PRISMA-P) statement which is registered in the International Platform of Registered Systematic Review and Meta-analysis Protocols PROSPERO (ID- CRD42021274925). We will perform an electronic search in MEDLINE, Embase, Scopus, PubMed, EBSCO, Web of Science, Global Index Medicus, Trip, and Cochrane Library databases, and in Google Scholar, using a customized search string. Any article that discusses any type of CL- and/or MCL-associated stigma in English, Spanish and Portuguese will be included. Articles targeting veterinary studies, sandfly vector studies, laboratory-based research and trials, articles focusing only on visceral leishmaniasis, and articles on diagnostic or treatment methods for CL and MCL will be excluded. Screening for titles and abstracts and full articles and data extraction will be conducted by two investigators. The risk of bias will be assessed through specific tools for different study types. A narrative synthesis of evidence will then follow. This review will identify the knowledge gap in CL-associated stigma and will help plan future interventions.

Introduction

Leishmaniasis is a tropical parasitic disease endemic in 98 countries across the globe including parts of Southern Europe, the Middle East, Africa, Asia, and South America [1]. The disease affects around 700,000–1,000,000 people every year [2]. Leishmaniasis is caused by a protozoan parasite, Leishmania spp. It is transmitted to humans by the bite of a female sandfly (genera Phlebotomus and Lutzomyia). Cutaneous leishmaniasis (CL), mucocutaneous leishmaniasis (MCL), and visceral leishmaniasis (VL) are the three main clinical types of this disease. CL is characterized by ulcerated skin lesions or non-ulcerative nodules. The size and the severity of the lesions vary [1]. MCL can be present as various manifestations such as lesions in the nasal and oral cavity, lesions in the subglottic region, trachea, and vocal cords. Some might even lead to the destruction of face structure, and in extreme cases even death [3, 4]. The prevalence of CL and MCL is increasing in many regions of the world, including Europe, the Middle East, Northern Africa, Asia, and parts of South America, and is recognized as a disease of the poor [58]. As these two manifestations affect bodily appearance, the stigma related to these conditions is likely to differ from that for visceral leishmaniasis.

Definitions and dimensions and mechanisms of stigma

Ervin Goffman pioneered research on stigma. In his book Stigma: Notes on the management of spoiled identity, 1963 he describes stigma as ‘‘the situation of the individual who is disqualified from full social acceptance” [9]. Stigma is defined in various ways, across disciplines. In their book: The Dilemma of Difference, researchers Ainlay, Becker, and Coleman propose a new definition of stigma as “a characteristic of persons that is contrary to a norm of a social unit. The characteristic may involve what people do (or have done), what they believe or whom they are (owing to physical or social characteristics)” [10]. Katz, in 1982, explains that “Stigmatized individuals have attributes that do not accord with the prevailing standards of the normal and good. They are often denigrated and avoided openly in the case of known criminals and other transgressors, or covertly and even unconsciously” [11]. In 2001, Link and Phelan discuss reasons for the variations in the definition of stigma; including the concept being applied to a multitude of unique circumstances, research on stigma being multidisciplinary, and researchers approaching the concept of stigma from different theoretical angles leading to different conceptualizations [12].

Not only the definition but also the classification and types of stigma vary according to the discipline of study and the disease under investigation [12]. Weiss describes three major types of stigma as enacted, anticipated, and internalized related to neglected tropical diseases (NTDs) [13]. Scambler and Hopkins define felt stigma as refers principally to the fear of enacted stigma. “a feeling of shame associated with a particular disease”. According to them enacted stigma is “instances of discrimination against people on the grounds of their perceived unacceptability or inferiority” [14]. Weiss defines Internalized stigma as “when a person with a stigmatized condition accepts perceived exclusionary views of society and self-stigmatizes himself or herself” [13]. With regard to tuberculosis, six types of stigma have been put forward: public stigma, anticipated or perceived stigma, secondary stigma, internalized or self-stigma, enacted or experienced stigma, and structural stigma [15].

Health-related stigma

Health-related stigma is described as the “social disqualification of individuals and populations who are identified with particular health problems” [16]. Health-related stigma leads to poor quality of life and mental health issues are a challenge to public health and a barrier to health interventions. Stigma can lead to poor prognosis, increased risk of disability, delay in diagnosis and treatment, and continuing risk of disease transmission, resulting in increased morbidity as well as psychological problems such as depression, stress, and fear, [1618]. Link, Phelan, and Hatzenbuehler argue that stigma should be considered a fundamental cause of health inequalities [19]. Structural social determinants such as racism, gender, and poverty can compound and aggravate stigma and restrict access to healthcare leading to inequities in health [20]. When socio-economics, politics, and inequalities that emerge from those structures are ignored, physicians can potentially misdiagnose patients and do real measurable harm [21]. Many studies identify social factors as the root of health inequalities. This highlights the importance of understanding and measuring stigma to assess the burden and implications of a particular disease and inform health system policy recommendations [6]. Furthermore, Hatzenbuehler emphasizes the need for more research to link measures of structural stigma to individual levels. As structural stigma is complex and needs an interdisciplinary approach to address, he emphasizes the need for future research that incorporates different fields such as sociology, anthropology, and epidemiology, with diverse methods and approaches [19, 22].

CL-associated stigma

Several studies have investigated the stigma associated with CL and how it disproportionately affects socio-economically deprived populations including women and children [23, 24]. Diverse overlapping classifications are available for CL-associated stigma. According to Al-Kamel, three types of CL-associated stigma are prevalent in society; CL-related social stigma, CL aesthetic stigma, and CL psychological stigma [25]. CL-related social stigma leads to the negative beliefs and perceptions that society has about people with the condition, whereas CL aesthetic stigma is explained as the dissatisfaction occurring from changes to the bodily image (scars, lesions, deformities) resulting from the condition. CL psychological stigma is the deep psychological stress that arises from experiencing the CL-related social stigma, CL aesthetic stigma [25]. Other potential types of stigma commonly associated with skin-related NTDs are likely to exist among CL patients and need to be further explored, including felt stigma, the expectation of discrimination because of CL; enacted stigma, and the unfair treatment from others towards CL patients [2628]. Stigma does not affect everyone equally and depends on a range of factors. There is a gap in knowledge around stigma associated with CL and MCL in particular when compared to other diseases such as leprosy and tuberculosis [15, 29, 30].

Measuring stigma

Although many interventions have been conducted to reduce health-related stigma, the effectiveness of these remains uncertain [16]. One explanation is the lack of validated tools to evaluate the impact of such interventions and accurately measure changes in the levels of stigma before and after the interventions. Link et al. emphasized the need for well-developed instruments to carry out stigma-related research. The observation and measurement of stigma are essential for the scientific understanding of the concept [31] and the need for in-depth studies on the stigma associated with CL is highlighted in the literature [32]. While mental health effects and the psychosocial burden of CL have been reviewed [3234], evidence synthesis focused on the stigma associated with CL and MCL and its implications as a whole has not been generated to date. This study aims to systematically review the dimensions, methods of measurement, implications, and preventive strategies of the stigma associated with CL.

Methods

This study has been registered on the International Platform of Registered Systematic Review and Meta-analysis Protocols (PROSPERO) under the registration number CRD42021274925. The systematic review protocol is prepared according to the guidelines given by the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) statement [35, 36] and the checklist is presented (S1 File).

Objectives

This review aims to systematically explore the stigma associated with CL and MCL and to identify related knowledge gaps.

This review seeks to address the following questions:

  1. What are the types of stigma associated with CL and MCL?
  2. What are the means of measuring stigma associated with CL and MCL and the different tools used?
  3. What are the implications of the stigma associated with CL and MCL, on patients and their families?
  4. What are the preventive strategies/interventions developed to address the stigma associated with CL and MCL?

Search strategy

A systematic search of published literature will be conducted using electronic databases, namely; MEDLINE, Embase, Scopus, PubMed, EBSCO, Web of Science, Global Index Medicus, Cochrane Library databases, and Google Scholar. Articles up to January 2023 will be considered in the search. The reference lists of included studies will be hand-searched to identify any additional evidence and potential sources of information. The articles will be searched using titles, abstracts, and keywords. The search strategy will be built under two main search term categories; terms describing stigma and terms describing the disease. The keywords were selected based on literature and after consulting the experts in leishmaniasis both locally and internationally.

Stigma*, discriminat*, stereotyp*, negative attitude, psychosocial impact, psychosocial burden, social consequences, stigmatizing effect, scar*, disfigure*, self-stigma, label avoidance, disgrace, shame, perception, and rejection

The keywords to select CL-related articles will be:

Dermal leishman*, cutaneous leishman*, oriental sore, Uta, Chiclero ulcer, tropical sore, Bagdad boil, Baghdad boil, Bauer ulcer, Delhi boil, Aleppo boil, Aleppo button, Jericho boil, one year sore, one year ulcer, tegumentary leishmaniasis, Biskra button, Biskra nodule, Calcutta ulcer, Jericho button, Kandahar sore, Lahore sore, Oriental button, Pian bois and Old World leishmaniasis.

The keywords to select MCL-related articles will be: Mucosal Leishman*, Mucocutaneous Leishman*, muco cutaneous Leishman*, espundia, nasopharyngeal Leishman*, New World leishmaniasis, American leishmaniasis, leishmaniasis americana

The final search string will be created as a combination of these key term categories. PubMed search results are provided as supporting documents (S2 File).

Two independent reviewers will carry out the whole process of systematic searches. Any disagreements will be solved by a discussion with the help of a third reviewer. A consistent decision will be reached after a discussion.

Inclusion criteria.

  • All articles which discuss any type of CL- and MCL-associated stigma entirely or partially
  • Articles in English, Spanish and Portuguese only
  • Qualitative studies, quantitative studies, cross-sectional studies, mixed-method studies, cross-sectional, grounded theory, ethnographic/anthropological, survey, case studies, and multidisciplinary studies will be included
  • Original articles, literature reviews, workshop abstracts, conference abstracts, conference presentations

Exclusion criteria.

  • Articles targeting veterinary studies, vector studies, laboratory-based research, and trials
  • Articles that explore stigma only in visceral leishmaniasis
  • Articles focused on diagnostic or treatment methods for CL and MCL
  • Articles which are focused on skin lesions caused by PKDL

Study flow and data management

First, duplications will be removed from the search results and the studies to be included will be first screened by title, abstract, and keywords. The investigators will review titles and abstracts to remove the obviously irrelevant studies and later in the second screening stage, full texts will be screened. The reason for the exclusion of articles retrieved will be documented. Then the full text will be screened and articles will be selected. Two reviewers will conduct the whole process independently and any disagreements will be resolved through discussion with a third reviewer.

Data extraction

A pre-piloted data extraction form will be used to extract data from the full text of all screened eligible articles. The data extraction form will include:

  1. Title
  2. Authors
  3. Year of publication
  4. Country/region
  5. Study setting and study design
  6. Reported demographics of participants
  7. Time period
  8. Method of data collection
  9. Stigma type
  10. Stigma scales used
  11. Characteristics of scales used
  12. Validity and reliability of scales
  13. Accuracy parameters of the scales used
  14. Any other conceptualization of stigma used
  15. Outcomes and impacts of the study
  16. Limitations of the study
  17. The conclusion of the study
  18. Recommendations/Future directions

Two individual investigators will collect data independently. If there is missing information, the investigators will contact the corresponding authors of those particular studies through the correspondents’ email provided within the article. Any contradictions in the data extraction will be resolved by discussion with the help of another author, and a final consistent decision will be made. The data extraction form is displayed as supporting documents (S3 File).

Data synthesis

Since the review uses both quantitative and qualitative studies, the analysis will accommodate both quantitative and descriptive approaches. Summary tables of all studies that are included will be produced including information such as title, publication year, the country in which the study was conducted, and a summary of findings. A narrative synthesis will be completed using the integrative review method. This method is used to summarize theoretical and empirical literature to convey a thorough understanding of a studied problem [37]. Due to the nature of the review questions, heterogeneity of included articles is expected. However, since a meta-analysis would not be conducted and the findings will be described for each review question separately as mentioned above this heterogeneity will not account for an error. Investigators will pay attention to heterogeneity in interpreting the findings in the narrative synthesis of evidence and discuss the evidence acknowledging the heterogeneity expected.

Assessment of risk of bias

Study quality of observational cohorts, cross-sectional studies, case-control studies, case series studies, and controlled intervention studies will be assessed by the study quality assessment toolset of the National Heart, Lung and Blood Institute (Study Quality Assessment Tools | NHLBI, NIH, 2021). The risk of bias in randomized control trials will be assessed by the revised Cochrane tool for assessing the risk of bias in randomized control trials (RoB tool) [38, 39]. All the other types of studies will be assessed using the Mixed Methods Appraisal Tool (MMAT). The MMAT was developed to appraise the quality of empirical studies and allows assessment of the methodological quality of quantitative, qualitative, and mixed methods studies [39]. We will assess qualitative studies for credibility, transferability, dependability and confirmability [40]. The study quality of each eligible study will be examined by two independent authors. The authors will indicate the appropriateness of the study for particular criteria, required by the tools. If there’s a major flow in any of the criteria that will be noted and later discussed. After individual assessments, each author will give a score to each article and then a final score will be decided after discussion. Any queried articles will be reviewed by a third author and a final consistent decision will be made. We will present the identified strengths and weaknesses with regard to the quality of the study for the selected studies for synthesis and will discuss the implications of the quality of studies in the generation of evidence with regard to stigma in CL and MCL [41].

Discussion

The topic of stigma is increasingly being considered across disciplines such as public health, mental health, medical anthropology, and sociology. However, stigma related to NTDs is relatively under-researched, especially for emerging infectious diseases such as CL and MCL in countries like Sri Lanka. CL-associated stigma is poorly understood, which hinders the ability to produce interventions to reduce stigma or address stigma-related issues. It is important to understand the existing knowledge on the stigma associated with CL as well as the implications of stigma on CL patients’ and their family members’ lives. Identification of the impacts of CL-associated stigma will be critical to informing the development of future preventive strategies and interventions. A review of the tools available to measure stigma will facilitate the development process of a gold standard tool to assess stigma in CL and similar NTDs. A narrative synthesis of different aspects of stigma related to the disease will be helpful in conceptualizing the concept of stigma itself and also identifying the drivers of stigma in CL and MCL informing public health experts and policymakers to plan appropriate interventions. This review will also highlight the geographical variations and commonalities of stigma in CL which will be helpful in planning global health interventions to reduce stigma and the psychosocial burden related to CL and MCL. In this study, we are not conducting a meta-analysis. Given the anticipated nature of the heterogeneity of the articles, tools, cultural influences and subjective nature of the concept we believe it will be unfeasible to conduct a meta-analysis. Another limitation of the study is that we will be not including publications in conference proceedings and other grey literature due to time constraints and lack of practicality. There could be other articles from other languages beyond Spanish, Portuguese and English which will not be used in this study which is another limitation.

This review will collate the knowledge on the stigma associated with CL from across the world and will be pivotal to understanding the existing evidence gaps for further exploration, and for the development and implementation of interventions to reduce stigma and related implications.

Supporting information

S2 File. Search conducted in PubMed on 17th January 2023.

https://doi.org/10.1371/journal.pone.0285663.s002

(PDF)

S3 File. Form to extract data from included studies.

https://doi.org/10.1371/journal.pone.0285663.s003

(PDF)

References

  1. 1. Reithinger R, Dujardin J, Louzir H, Pirmez C, Alexander B, Brooker S, et al. Cutaneous leishmaniasis. Lancet Infect Dis. 2007;7: 581–596. pmid:17714672
  2. 2. Alvar J, Vélez ID, Bern C, Herrero M, Desjeux P, Cano J, et al. Leishmaniasis worldwide and global estimates of its incidence. PLoS One. 2012;7. pmid:22693548
  3. 3. Zajtchuk JT, Casler JD, Netto EM, Grogl M, Neafie RC, Hessel CR, et al. Mucosal leishmaniasis in Brazil. Laryngoscope. 1989;99: 925–939. pmid:2671555
  4. 4. Carvalho EM, Llanos-Cuentas A, Romero GAS. Mucosal leishmaniasis: Urgent need for more research. Rev Soc Bras Med Trop. 2018;51: 120–121. pmid:29513835
  5. 5. Strazzulla A, Cocuzza S, Pinzone MR, Postorino MC, Cosentino S, Serra A, et al. Mucosal leishmaniasis: An underestimated presentation of a neglected disease. Biomed Res Int. 2013;2013. pmid:23853773
  6. 6. Marmot M. Social determinants of health inequalities. Lancet. 2005;365: 1099–1104. pmid:15781105
  7. 7. Alvar J, Yactayo S, Bern C. Leishmaniasis and poverty. Trends Parasitol. 2006;22: 552–557. pmid:17023215
  8. 8. Burza S, Croft SL, Boelaert M. Leishmaniasis. The Lancet. 2018;392: 951–970. pmid:30126638
  9. 9. Goffman E. Stigma: Notes on the Management of Spoiled Identity. New York: Simon & Schuster; 1963.
  10. 10. Ainlay SC, Becker G, Coleman LM. The Dilemma of Difference—A Multidisciplinary View of Stigma. 1st editio. Aronson E, editor. Plenum press, New York; 1986.
  11. 11. Katz I. Stigma: a social psychological analysis. Erlbaum Associates Hillsdale, N.J; 1981.
  12. 12. Link BG, Phelan JC. Conceptualizing Stigma. Annu Rev Sociol. 2001;21: 363–385.
  13. 13. Weiss MG. Stigma and the Social Burden of Neglected Tropical Diseases. 2008;2. pmid:18478049
  14. 14. Scambler G, Hopkins A. Being epileptic: coming to terms with stigma. Sociol Health Illn. 1986;8: 26–43.
  15. 15. Mitchell EM, van den Hof Authors Agnes Meershoek S, Zwerling A, Daftary A, Citro B, Smyth C, et al. TB Stigma. 2018.
  16. 16. Weiss MG, Ramakrishna J, Somma D. Health-related stigma: Rethinking concepts and interventions. Psychol Health Med. 2006;11: 277–287. pmid:17130065
  17. 17. Van Brakel WH. Measuring health-related stigma—a literature review. Psychol Health Med. 2006;11: 307–334. pmid:17130068
  18. 18. Kane JC, Elafros MA, Murray SM, Mitchell EMH, Augustinavicius JL, Causevic S, et al. A scoping review of health-related stigma outcomes for high-burden diseases in low- and middle-income countries. BMC Med. 2019;17. pmid:30764819
  19. 19. Hatzenbuehler ML, Phelan JC, Link BG. Stigma as a fundamental cause of population health inequalities. Am J Public Health. 2013;103: 813–821. pmid:23488505
  20. 20. Farmer PE, Nizeye B, Stulac S, Keshavjee S. Structural violence and clinical medicine. PLoS Med. 2006;3: 1686–1691. pmid:17076568
  21. 21. Holmes SM, Helena H, Angela J, Scott D. S, Michelle M, Jeremy A. G, et al. Misdiagnosis, Mistreatment, and Harm—When Medical Care Ignores Social Forces. N Engl J Med. 2020;382: 1083–1086. pmid:32187466
  22. 22. Hatzenbuehler ML. Structural Stigma and Health. The Oxford Handbook of Stigma, Discrimination, and Health. 2018. pp. 1–28.
  23. 23. Reithinger R, Aadil K, Kolaczinski J, Mohsen M, Hami S. Social impact of leishmaniasis, Afghanistan. Emerg Infect Dis. 2005;11: 634–636. pmid:15834984
  24. 24. Hotez PJ. Stigma: The stealth weapon of the NTD. PLoS Negl Trop Dis. 2008;2: 1–2. pmid:18446206
  25. 25. Al-kamel MA. Stigmata in cutaneous leishmaniasis: Historical and new evidence-based concepts. 2017;8: 81–90.
  26. 26. Marahatta SB, Amatya R, Adhikari S, Giri D, Lama S, Kaehler N, et al. Perceived stigma of leprosy among community members and health care providers in Lalitpur district of Nepal: A qualitative study. PLoS One. 2018;13: 1–13. pmid:30589875
  27. 27. Deribe K, Tomczyk S, Mousley E, Tamiru A, Davey G. Stigma towards a Neglected Tropical Disease: Felt and enacted Stigma Scores among Podoconiosis Patients in Northern Ethiopia. BMC Public Health. 2013;13. pmid:24330684
  28. 28. de Zeeuw J, Douwstra M, Omansen TF, Sopoh GE, Johnson C, Phillips RO, et al. Psychometric Properties of the Participation Scale among Former Buruli Ulcer Patients in Ghana and Benin. PLoS Negl Trop Dis. 2014;8. pmid:25393289
  29. 29. da Morgado FFR, da Silveira EMKX, Sales AM, do Nascimento LPR, Sarno EN, da Nery , et al. Cross-cultural adaptation of the EMIC Stigma Scale for people with leprosy in Brazil. Rev Saude Publica. 2017;51: 80. pmid:28876410
  30. 30. van Brakel WH, Sihombing B, Djarir H, Beise K, Kusumawardhani L, Yulihane R, et al. Disability in people affected by leprosy: the role of impairment, activity, social participation, stigma and discrimination. Glob Health Action. 2012;5: 1–11. pmid:22826694
  31. 31. Link BG, Yang LH, Phelan JC, Collins PY. Measuring mental illness stigma. Schizophr Bull. 2004;30: 511–541. pmid:15631243
  32. 32. Bennis I, De Brouwere V, Belrhiti Z, Sahibi H, Boelaert M. Psychosocial burden of localised cutaneous Leishmaniasis: A scoping review. BMC Public Health. 2018;18. pmid:29544463
  33. 33. Hofstraat K, Van Brakel WH. Social stigma towards neglected tropical diseases: a systematic review. 2016;8: 53–70. pmid:26940310
  34. 34. Id MP, Wright B, Id PMK. The impact of leishmaniasis on mental health and psychosocial well-being: A systematic review. 2019; 1–21. pmid:31622369
  35. 35. BMJ. PRISMA 2020 explanation and elaboration: Updated guidance and exemplars for reporting systematic reviews. The BMJ. 2021;372. pmid:33781993
  36. 36. Hutton B, Salanti G, Caldwell DM, Chaimani A, Schmid CH, Cameron C, et al. The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: Checklist and explanations. Ann Intern Med. 2015;162: 777–784. pmid:26030634
  37. 37. Whittemore R, Knafl K. The integrative review: Updated methodology. J Adv Nurs. 2005;52: 546–553. pmid:16268861
  38. 38. Sterne JAC, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, et al. RoB 2: A revised tool for assessing risk of bias in randomised trials. The BMJ. 2019;366: 1–8. pmid:31462531
  39. 39. Hong Q, Pluye P, Fàbregues S, Bartlett G, Boardman F, Cargo M, et al. Mixed Methods Appraisal Tool (MMAT), Version 2018. User guide. McGill. 2018; 1–11.
  40. 40. Hannes K. Chapter 4: Critical appraisal of qualitative research. Version 1. In: Noyes J, Booth A, Hannes K, Harden A, Harris J, Lewin S, et al., editors. Supplementary Guidance for Inclusion of Qualitative Research in Cochrane Systematic Reviews of Interventions. Version 1. Cochrane Collaboration Qualitative Methods Group; 2011. http://cqrmg.cochrane.org/supplemental-handbook-guidance
  41. 41. Pace R, Pluye P, Bartlett G, Macaulay AC, Salsberg J, Jagosh J, et al. Testing the reliability and efficiency of the pilot Mixed Methods Appraisal Tool (MMAT) for systematic mixed studies review. Int J Nurs Stud. 2012;49: 47–53. pmid:21835406