https://orcid.org/0000-0002-4375-3729
Update
6 Feb 2023: Gabriel FC, Stein AT, Melo DdO, Fontes-Mota GCH, dos Santos IB, et al. (2023) Guidelines’ recommendations for the treatment-resistant depression: A systematic review of their quality. PLOS ONE 18(2): e0281501. https://doi.org/10.1371/journal.pone.0281501 View update
Figures
Abstract
Introduction
Depression is a serious and widespread mental health disorder. Although effective treatment does exist, a significant proportion of patients with depression fail to respond to antidepressant treatment trials, a condition named treatment-resistant depression. Efficient approach should be given this condition in order to revert the burden caused by depression. Clinical practice guidelines (CPGs) are evidence-based health promotion instruments to improve diagnosis and treatment. CPGs recommendations for treatment-resistant depression must be trustworthy. The objective of the proposed study is to systematically identify, appraise the quality of CPGs for the treatment of depression and elaborate a synthesis of recommendations for treatment-resistant depression of CPGs considered to be of high quality and with high quality recommendations.
Methods and analysis
We will search the databases of organizations, such as PubMed, Embase, Cochrane Library, PsycInfo, and the Virtual Health Library, and organizations that develop CPGs. Three independent researchers will assess the quality of the CPGs and their recommendations using the AGREE II and AGREE-REX instruments, respectively. Given the identification of divergences and convergences as well as weak and strong points among high quality CPGs, our work may help developers, clinicians and eventually patients.
Citation: Gabriel FC, Stein AT, de Melo DO, Fontes-Mota GCH, dos Santos IB, Rodrigues CdS, et al. (2022) Recommendations for the pharmacological treatment of treatment-resistant depression: A systematic review protocol. PLoS ONE 17(4): e0267323. https://doi.org/10.1371/journal.pone.0267323
Editor: Maria G. Grammatikopoulou, International Hellenic University: Diethnes Panepistemio tes Ellados, GREECE
Received: October 15, 2021; Accepted: April 6, 2022; Published: April 19, 2022
Copyright: © 2022 Gabriel et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: Initials of the authors who received each award: FCG Award Number: 141811/2020-0 The full name of each funder: Conselho Nacional de Desenvolvimento Científico e Tecnológico URL of each funder website: https://www.gov.br/cnpq/pt-br Have or will the sponsors or funders play any role in the study design, data collection (when applicable) and analysis, decision to publish, or preparation of the manuscript? The funders had not and will not have a role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Initials of the authors who received each award: FCG.
Competing interests: Ivan Florez is the current leader in the AGREE collaboration. All other authors declare that they have no conflicts of interest. This does not alter our adherence to PLOS ONE policies on sharing data and materials.
Introduction
Depression is a serious medical illness that negatively affects behaviour. This condition causes feelings of sadness and/or loss of interest in everyday life activities. It is also a common condition, which affects more than 300 million people worldwide and is considered one of the most relevant public health problems in the 21st century [1]. Owing to its disabling nature, it can cause various professional, economic, social, and personal losses [2]. Patients with lower response to depression treatment have higher risk of severe outcomes including job loss, isolation, and suicide which may lead to an increased economic cost to society [3]. In addition, over the last years the number of depressed people has increased considerably, overloading health systems and generating a greater need for resource optimization [4].
There are several classes of antidepressants including serotonin reuptake inhibitors, serotonin and norepinephrine reuptake inhibitors, tricyclic antidepressants, noradrenergic and serotonergic antidepressants, monoamine oxidase inhibitors, and others, which are effective for medical treatment of depression [5]. Around only one third of patients will remit after a trial with a selective serotonin reuptake inhibitor (SSRI) and only 25% to 27% will remit after a subsequent trial with another antidepressant [6]. Consequently, a significant number of patients—up to 40%—will be treatment-resistant (i.e., failed to remit to at least two antidepressant trials) [6].
Treatment-resistant depression is difficult to manage, and results are usually poor, especially when unstandardized approaches are used [7–10]. In this respect, clinical practice guidelines (CPGs) are an essential tool for guiding clinical decisions. CPGs are supported by the best available evidence gathered through rigorous systematic reviews. In addition, CPGs consider additional key elements before providing a recommendation—such as costs, feasibility, patients’ values, and preferences—and aim to balance benefits and harms, among others key aspects [11, 12].
Several instruments are available to evaluate the methodological quality of CPGs [13, 14]. One of the most widely used is the Appraisal of Guidelines Research & Evaluation (AGREE II). This instrument provides a comprehensive, rapid, and robust assessment of CPGs [13, 15, 16]. The quality of CPGs for depression has been the focus of several reviews in recent years. For instance, in a recent review, only 4 out of 11 CPGs met the minimum quality benchmark after their assessment with the AGREE II tool [17]. Another review found that only 6 out of 27 CPGs could be classified as high-quality CPGs [18].
The low methodological quality of CPGs can cause inconsistencies among recommendations [19, 20]. In a study comparing the recommendations for pharmacotherapy and neurostimulation in the treatment of depression, a high degree of inconsistency was found in the recommendations for the second and third lines of treatment [21]. Discrepancies among recommendations for first-line treatments and for patients not responding to first-line treatment were also observed by MacQueen et al. [20], in a study analyzing CPGs used in primary healthcare for major depression, dysthymia, and minor depression.
High quality CPGs are crucial for developing recommendations that are based on the best evidence, with a transparent and trustworthy process that is implementable and acceptable by stakeholders and patients. However, it should be noted that high-quality CPGs, i.e., CPGs that obtain high scores on AGREE II assessment, do not necessarily guarantee credible and implementable recommendations [22]. In response to this gap, the AGREE Collaboration developed another instrument to evaluate the quality of the recommendations. The Appraisal of Guidelines Research and Evaluation-Recommendations Excellence (AGREE-REX) tool [23, 24], includes additional factors—such as clinical application, stakeholder values and preferences, and feasibility of implementation. These factors are considered by CPG developers for more feasible, credible, and implementable recommendations, and by guideline users interested in evaluating recommendation quality.
Although previous studies have evaluated the methodological quality of CPGs for the treatment of depression using AGREE II and the discrepancies among recommendations [18, 25–27], no study has evaluated the quality of their recommendations. However, doing so would be important for the development of CPGs by increasing the reliability of the recommendations, which could enhance evidence-based clinical practices implementation. Finally, there is a need to have reliable CPGs for treatment-resistant depression, as an effective treatment is a very common decision-making question among clinicians, and much inconsistency occurs in clinical practice, as the available CPGs do not have a strong recommendation. The scope of this proposal is to fulfill this lack in the management of depression as a public health problem. Mainly because these are patients who, with each therapeutic failure of pharmacological treatments, are more likely to have serious outcomes such as suicide.
Thus, the aim of this systematic review protocol is to identify and appraise the certainty of evidence of recommendations to the approach of treatment-resistant depression in CPGs.
Materials and methods
Literature search
A comprehensive bibliographic search will be conducted across the following databases: MEDLINE (through PubMed), Embase, Cochrane Library, PsycInfo, and the Virtual Health Library. The complete database search strategy is presented in S1 Appendix. A manual search will also be undertaken on the websites of organizations responsible for developing or compiling CPGs such as the Australian Clinical Practice Guidelines (clinicalguidelines.gov.au), the Brazilian Ministry of Health (saude.gov.br), the Canadian Agency for Drugs and Technologies in Health (cadth.ca), the Chilean Ministry of Health (librarinsal.cl/guides-clinicasauge/), the Colombian Ministry of Health and Social Protection (http://gpc.minsalud.gov.co/gpc/SitePages/default_gpc.aspx), the Guidelines International Network (https://guidelines.ebmportal.com/), the Institute for Clinical Systems Improvement (icsi.org), the GuíaSalud Portal (Spain) (guiasalud.es), the Scottish Intercollegiate Guidelines Network (sign.ac.uk), the National Institute for Health and Care Excellence (nice.org.uk), Guideline Central (https://www.guidelinecentral.com/library/), and the ECRI library (https://www.ecri.org/library/).
We will also refer to documents complementary to the identified CPGs, such as summaries of recommendations, documents intended for patient education, search strategies, and methodological manuals. The literature search will be limited to CPGs published in the last ten years due to new initiatives to understand and establish criteria to define and diagnose treatment-resistant depression.
Eligibility criteria
The CPGs providing recommendations for the treatment- resistant depression in adults will be included in the study. We will consider all documents that provide recommendations for the pharmacological management of depression, regardless of the methodology used to develop the recommendations. We will exclude CPGs intended only for local use, (e.g., developed to be used in a hospital), or specifically for patients with comorbidities (e.g., CPGs for patients with cancer, dementia, or multiple sclerosis) due to the lack of certainty of evidence and inability to apply in these different contexts. Moreover, CPGs focused on nonpharmacological interventions for depression will also be excluded (e.g., psychotherapy and electroconvulsive therapy). We will include CPGs published in the last ten years. In case of two or more versions of the same CPG in this period, we will include only the updated version.
Protocol registration.
The results of this study will be published in the form of a scientific paper that will be prepared according to the PRISMA 2020 guidelines [28].
Selection of CPGs
Fig 1 illustrates the planned CPG selection process. Duplicates will be removed using EndNote and the final reference list will be imported to the online review platform Rayyan® [29] for the screening and selection process.
The selection of CPGs will be conducted in two steps. First, titles and abstracts will be independently screened by two reviewers to verify their eligibility. Then, the two reviewers will also independently appraise the full text of potentially eligible documents. Discrepancies between reviewers in those two steps will be discussed and resolved by consensus, or by a third researcher if consensus is not achieved.
Data extraction
Data extraction from the selected CPGs will be performed independently by two researchers using a piloted standard form on Google Forms®. The available data will be downloaded as an Excel spreadsheet and checked for agreement. Discrepancies will be resolved by consensus among the researchers and by a third researcher if consensus is not achieved.
The following data will be extracted from each included CPG: year of publication or updated version of the CPGs, classification of evidence, and institution or organization.
Evaluation of the quality of the CPGs
The quality of the CPGs will be evaluated using the AGREE II instrument. The Portuguese version of the tool will be used [30]. It comprises 23 items organized into six domains: 1) scope and purpose, 2) stakeholder involvement, 3) rigour of development, 4) clarity of presentation, 5) applicability, and 6) editorial independence [14]. Each AGREE II item is scored using a 7-point Likert scale, where 1 indicates that there is no information for the AGREE II item and 7 indicates that the information is of the highest possible quality. Three reviewers (FCG, GCHFM, and IBS) will perform the quality assessment. More information regarding reviewer training is provided in the training subsection below.
An experienced investigator will train the reviewers using the AGREE II tool. Discrepancies between AGREE II items (2 points or more between reviewers) will be discussed among reviewers until a consensus is reached. If no consensus is reached, the disagreement will be resolved by a fourth senior researcher. Final scores per domain will be calculated by summing the scores from the three researchers per individual item in a domain and by scaling the total as a percentage of the maximum possible score for that domain. Therefore, each domain will be scored in a range from 0% to 100% [30].
There is no established threshold to consider a CPG of high quality according to AGREE II [24]. However, we considered that CPGs with a score higher than 60% in domain 3 (rigour of development) of AGREE II will be classified as high-quality, since this cut-off point has already been adopted by several authors [27, 31–33].
The quality assessment scores will be automatically calculated for each domain using the AGREE Plus Platform® [34]. According to the Agree II users’ Manual [30], the scores for each domain will be regarded as independent. Moreover, since some authors have highlighted how variable the overall scores in AGREE II can be and the manual recommends using the domains independently, we will not use the final overall domain for the analyses [25, 35].
Evaluation of the quality of recommendations
The AGREE-REX tool will be used to assess the quality of the CPG recommendations [24]. We will apply the tool to all the pharmacological recommendations provided by the CPGs. AGREE-REX consists of nine items grouped into three domains: clinical applicability, values and preferences, and feasibility of implementation. Each item will be scored independently by three reviewers, using a 7-point Likert scale. When a discrepancy of 2 points or more occurs among evaluators, we will try to resolve it through discussion and consensus. If no consensus is reached, a fourth reviewer will decide the final score. Each appraiser’s notes will be inserted into a Google form®, and the results will be saved in a Microsoft Excel spreadsheet. Similar to the approach described for AGREE II, the final scores per domain will be obtained with values between 0 and 100% [34]. Recommendations with scores higher than 60% in domain 1 (clinical applicability) of the AGREE-REX will be considered as of high-quality using the same criteria considered for the quality of the CPGs. Each AGREE-REX question will receive scores from the three appraisers, final score will be defined by consensus in case of discrepancies, and a fourth, senior researcher will make a decision.
AGREE II and AGREE-REX training
We will develop a training process based on the approaches described above [36–38]. The training will include a detailed discussion of the AGREE II and AGREE-REX manuals with instructions for their use and key articles that have used those methodologies [23]. The researchers will later make the training available on the AGREE Research Trust (2017) website [39]. As part of the training, the evaluators will assess the quality of CPGs for the treatment of chronic pain [40], Gaucher’s disease [41], and obesity [42]. We used the evaluation of these CPGs to calibrate the AGREE instruments. After the assessment, reviewers and trainers will discuss the scores and the discrepancies found. The trainers will provide feedback and answer questions posed by the reviewers. In the second stage, the team will evaluate two of the latest CPGs for hyperthyroidism and urinary tract infection [43, 44]. Later, a second discussion about the tool and discrepancies will be held [27].
Strategy for data synthesis
Only CPGs considered to have high-quality (AGREE II domain 3 score ≥ 60%) with recommendations also categorized as high-quality (AGREE-REX domain 1 score ≥ 60%) will be the included in the data synthesis.
Domain 3 from AGREE II (rigour of development) will be used to classify a CPG as “high-quality”, since this is the most important item regarding the reliability of its recommendations [45]. Domain 1 from AGREE-REX will be used specifically to classify recommendations as having high quality. For the extraction of recommendations from the selected CPGs only two independent researchers will be used. This task will include extracting the quality of the evidence and strength of the recommendations. We will group the recommendations in two main topics, “terminology for resistant depression” and “recommended management strategies”. The terminologies and sequences of the therapeutic strategies will be compared between the CPGs and the strategies and terminologies that the CPGs had in common will be synthesized in a table.
Discussion
With the identification of divergences and convergences between the recommendations of CPGs, as well as the identification of flaw points of CPGs for the pharmacological treatment of treatment-resistant depression we will be able to reveal areas deserving improvements. Such improvements will help developers of CPGs to enhance their processes and create both superior quality CPGs and recommendations. In addition, this will also increase the likelihood that the CPGs will be implemented, which will benefit patients as well as healthcare professionals.
This protocol has several strengths. First, we will conduct a comprehensive literature search in 17 databases and CPG repositories. Second, appraisers will undergo rigorous training in the AGREE II and AGREE-REX tools. This will increase the reliability of the results. Third, assessments will be performed independently by three researchers, minimizing the subjectivity of the intrinsic evaluation method and thus, improving reliability. The strengths and limitations of each CPG will be thoroughly evaluated during the data collection and analysis process.
However, the study had some limitations as well. Treatment of depression requires a multidimensional approach, not only pharmacological treatment. We only plan to assess CPGs on pharmacological treatment, and therefore, will not be able to draw conclusions for CPGs covering exclusively other treatment alternatives (e.g., psychotherapy and electroconvulsive therapy). Furthermore, although we are planning to perform manual searches of many specific websites, we cannot guarantee that relevant CPGs are not missed.
Nevertheless, these possible limitations do not lessen the relevance of the proposed protocol, particularly considering the comprehensive literature search, rigorous training in the AGREE II and AGREE-REX tools, and assessment by multiple researchers in order to adequately appraise the certainty of evidence of CPGs addressing the management of treatment-resistant depression.
Supporting information
S1 Appendix. Complete database search strategy.
https://doi.org/10.1371/journal.pone.0267323.s001
(DOCX)
Acknowledgments
We thank Caroline Molino, Luciana Vasconcelos, and Nathália Celini Leite Santos for their invaluable assistance in helping us better understand the method of appraisal of CPGs.
References
- 1.
World Health Organization. Depression and other common mental disorders: global health estimates [Internet]. Geneva: WHO; 2017 [cited 2017 Dec 8]. Available from: http://apps.who.int/iris/bitstream/10665/254610/1/WHO-MSD-MER-2017.2-eng.pdf
- 2.
Thompson C. Affective disorders: the instruments of psychiatric research. London: John Wiley & Sons; 2010.
- 3. Nemeroff CB. Prevalence and management of treatment-resistant depression. J Clin Psychiatry. 2007;68: 17–25. pmid:17640154
- 4. Cipriani A, Furukawa TA, Salanti G, Chaimani A, Atkinson LZ, Ogawa Y, et al. Comparative efficacy and acceptability of 21 antidepressant drugs for the acute treatment of adults with major depressive disorder: a systematic review and network meta-analysis. Lancet. 2018;391: 1357–66. pmid:29477251
- 5.
Taylor DM, Barnes TRE, Young AH. The Maudsley Prescribing Guidelines in Psychiatry. John Wiley and Sons, 13a Ed., 2018, 872p.
- 6. Rush AJ, Fava M, Wisniewski SR, et al. Sequenced treatment alternatives to relieve depression (STAR*D): Rationale and design. Control Clin Trials. Feb 2004;25(1):119–142. https://doi.org/10.1016/s0197-2456(03)00112-0 pmid:15061154
- 7. Dunner DL, Rush AJ, Russell JM, Burke M, Woodard S, Wingard P, et al. Prospective, long-term, multicenter study of the naturalistic outcomes of patients with treatment-resistant depression. J Clin Psychiatry. 2006;67: 688–95. pmid:16841617
- 8. Wooderson SC, Juruena MF, Fekadu A, Commane C, Donaldson C, Cowan M, et al. Prospective evaluation of specialist inpatient treatment for refractory affective disorders. J Affect Disord. 2011;131: 92–103. pmid:21144591
- 9. Fekadu A, Wooderson SC, Markopoulo K, Donaldson C, Papadopoulos A, Cleare AJ. What happens to patients with treatment-resistant depression? A systematic review of medium to long term outcome studies. J Affect Disord. 2009;116: 4–11. pmid:19007996
- 10. Fekadu A, Wooderson S, Donaldson C, Markopoulou K, Masterson B, Poon L, et al. A multidimensional tool to quantify treatment resistance in depression: the Maudsley staging method. J Clin Psychiatry. 2009;70: 177–84. pmid:19192471
- 11.
Institute of Medicine (US). Clinical practice guidelines we can trust. Washington, DC: National Academies Press; 2011.
- 12. Mota DM, Vigo A, Kuchenbeckr RS. [Evaluation of the performance of the brazilian notification system for health surveillance: a tool in Brazil’s pharmacovigilance system]. Cien Saude Colet. 2020;25: 1955–66. Portuguese. pmid:32402025
- 13. Siering U, Eikermann M, Hausner E, Hoffmann-EBer W, Neugebauer EA. Appraisal tools for clinical practice guidelines: a systematic review. PLoS One;8: e82915. pmid:24349397
- 14. Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, et al. AGREE II: advancing in the development, reporting and evaluation of guidelines in the area of health. CMAJ. 2010;182: E839–42. pmid:20603348
- 15. Semlitsch T, Blank WA, Kopp IB, Siering U, Siebenhofer A. Evaluating guidelines: a review of key quality criteria. Dtsch Arztebl Int. 2015;112: 471–8. pmid:26214233
- 16. Legido-Quigley H, Panteli D, Brusamento S, Knai C, Saliba V, Turk E, et al. Clinical guidelines in the European Union: mapping the regulatory base, development, quality control, implementation and evaluation in member states. Health Policy. 2012;107: 146–56. pmid:22939646
- 17. Zafra-Tanaka JH, Goicochea-Lugo S, Villarreal-Zegarra D, Taype-Rondan A. Characteristics and quality of clinical practice guidelines for depression in adults: a scoping review. BMC Psychiatry. 2019;19: 79. pmid:30791878
- 18. Gabriel FC, Melo DO, Fráguas R, Leite-Santos NC, Silva RAM, Ribeiro E. Pharmacological treatment of depression: a systematic review comparing clinical practice guideline recommendations. PLoS One. 2020;15: e0231700. pmid:32315333
- 19. Saturno Hernández PJ, Antón Botella JJ, Murcia Payá JF. Variability en la protocolización de la captación, diagnóstico y valoración inicial del paciente hipertenso en 40 centros de salud. Med Clin (Barc). 2000;114: 14–8. pmid:10916800.
- 20. Macqueen G, Santaguida P, Keshavarz H, Jaworska N, Levine M, Beyene J, et al. Systematic review of clinical practice guidelines for failed antidepressant treatment response in major depressive disorder, dysthymia, and subthreshold depression in adults. Can J Psychiatry. 2017;62: 11–23. pmid:27554483
- 21. Bayes AJ, Parker GB. Comparison of guidelines for the treatment of unipolar depression: a focus on pharmacotherapy and neurostimulation. Acta Psychiatr Scand. 2018;137: 459–71. pmid:29577229
- 22.
AGREE-REX Research Team. The appraisal of guidelines research & evaluation: recommendation excellence (AGREE-REX) [Internet]. [place unknown]: AGREE; 2019 [cited 2019 May 17]. Available from: https://www.agreetrust.org/wp-content/uploads/2019/04/AGREE-REX-2019.pdf
- 23. Brouwers MC, Spithoff K, Kerkvliet K, Alonso-Coello P, Burgers J, Cluzeau F, et al. Development and validation of a tool to assess the quality of clinical practice guideline recommendations. JAMA Netw Open 2020;3: e205535. pmid:32459354
- 24. Brouwers MC, Spithoff K, Lavis J, Kho ME, Makarski J, Florez ID. What to do with all the AGREEs? The AGREE portfolio of tools to support the guideline enterprise. J Clin Epidemiol. 2020;125: 191–7. pmid:32473992
- 25. Alonso-Coello P, Irfan A, Solà I, Gich I, Delgado-Noguera M, Rigau D, et al. The quality of clinical practice guidelines over the last two decades: a systematic review of guidelines evaluation studies. Qual Saf Health Care. 2010;19: e58. pmid:21127089
- 26. Kung J, Miller RR, Mackowiak PA. Failure of clinical practice guidelines to meet institute of medicine standards: two more decades of little, if any, prorgess. Arch Intern Med. 2012;172: 1628–33. pmid:23089902
- 27. Molino CGRC, Leite-Santos NC, Gabriel FC, Weinberg SK, Vasconcelos LP, Montovani-Silva RA, et al. Factors associated with high-quality guidelines for the pharmacologic management of chronic diseases in primary care: a systematic review. JAMA Intern Med. 2019;179:553–60. pmid:30776060
- 28. Page MJ, Mckenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372: n71. https://doi.org/10.1136/bmj.n71 pmid:33782057
- 29. Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan: a web and mobile app for systematic reviews. Syst Rev. 2016;5: 210. pmid:27919275
- 30.
Consórcio AGREE. Instrumento para avaliação de diretrizes clínicas. [place unknown]: AGREE; 2009 [cited 2021 06 29]. Available from: https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&ved=2ahUKEwiEiL6a0NL2AhUeupUCHdNUDZQQFnoECAgQAQ&url=https%3A%2F%2Fwww.agreetrust.org%2Fwp-content%2Fuploads%2F2013%2F06%2FAGREE_II_Brazilian_Portuguese.pdf&usg=AOvVaw2c8A5pFTt2dxe36g2ERT0g
- 31. Acuña-Izcaray A, Sánchez-Angarita E, Plaza V, Rodrigo G, Montes de Oca M, Gich I, et al. Quality assessment of asthma clinical practice guidelines: a systematic appraisal. Chest. 2013;144: 390–7. pmid:23450305
- 32. Arevalo-Rodriguez I, Pedraza OL, Rodríguez A, Sánchez E, Gich I, Solà I, et al. Alzheimer’s disease dementia guidelines for diagnostic testing: a systematic review. Am J Alzheimers Dis Other Demen. 2013;28: 111–9. pmid:23288575
- 33. Watine J, Friedberg B, Nagy E, Onody R, Oosterhuis W, Bunting PS, et al. Conflict between guideline methodologic quality and recommendation validity: a potential problem for practitioners. Clin Chem. 2006;52: 65–72. pmid:16391328
- 34.
Agreetrust.org [Internet]. [place unknown]: The AGREE Research Trust; c2010. Available from: https://www.agreetrust.org
- 35. Knai C, Brusamento S, Legido-Quigley H, Saliba V, Panteli D, Turk E, et al. Systematic review of the methodological quality of clinical guideline development for the management of chronic disease in Europe. Health Policy. 2012;107:157–16. pmid:22795610
- 36. Khan GSC, Stein AT. Adaptação transcultural do instrumento Appraisal of Guidelines for Research & Evaluation II (AGREE II) para avaliação de diretrizes clínicas. Cad Saude Publica. 2014;30: 1111–4. pmid:24936826
- 37. Colunga-Lozano LE, Gerardo-Morales V, Pérez-Gaxiola G, Vázquez-Álvarez AO, González-Torres FJ, Perales-Guerrero L, et al. Methodological assessment of Mexican clinical practice guidelines: GRADE framework adherence and critical appraisal. J Eval Clin Pract. 2021;27: 385–90. pmid:32779256
- 38. Ronsoni RM, Pereira CCA, Stein AT, Osanai MH, Machado CJ. Evaluation of eight clinical protocols and therapeutic guidelines (PCDT) of the ministry of health using the AGREE II instrument: a pilot study. Cad Saude Publica. 2015;31: 1157–62. pmid:26200364
- 39.
AGREE Next Steps Consortium. The AGREE II instrument [Internet]. [place unknown]: AGREE; 2017 [cited 5 May 2020]. Available from: http://www.agreetrust.org.
- 40.
Ministério da Saúde (BR). Portaria no. 1.083, de 2 de outubro de 2012. Aprova o protocolo clínico e diretrizes terapêuticas da dor crônica [Internet]. Brasília, DF: MS; 2012 [cited 2019 Jan 22]. Available from: https://bvsms.saude.gov.br/bvs/saudelegis/sas/2012/prt1083_02_10_2012.html
- 41.
Ministério da Saúde (BR). Portaria no. 1.266, de 14 de novembro de 2014. Aprova o protocolo clínico e diretrizes terapêuticas da doença de Gaucher. Brasília, DF: MS; 2014 [cited 2017 Apr 5]. Available from: https://bvsms.saude.gov.br/bvs/saudelegis/sas/2014/prt1266_14_11_2014.html.
- 42.
National Institute for Health and Care Excellence. Obesity prevention: clinical guideline [Internet]. [London]: NICE; 13 Dec 2006 [last updated 1 Mar 2015; cited 5 Apr 2017]. Available from: https://www.nice.org.uk/guidance/cg43/resources/obesity-prevention-975445344709.
- 43. Bahn RS, Burch HB, Cooper D, Garber JR, Greenlee MC, Klein I, et al. Hyperthyroidism and other causes of thyrotoxicosis: management guidelines of the American thyroid association and American association of clinical endocrinologists. Thyroid. 2011;21: 593–646. pmid:21510801
- 44.
Urinary Tract Infection Guideline Team. Urinary tract infection. Ann Arbor (MI): University of Michigan Health System; 2011 Jun.
- 45. Hoffmann-Eßer W, Siering U, Neugebauer EAM, Brockhaus AC, McGauran N, Eikermann M. Guideline appraisal with AGREE II: online survey of the potential influence of AGREE II items on overall assessment of guideline quality and recommendation for use. BMC Health Serv Res. 2018;18: 143. pmid:29482555