Class specific deprescribing guidelines could help clinicians taper and stop medications no longer needed or which may be causing more harm than benefit. We set out to develop methodology to create such guidelines using evidence-based methods for guideline development, evidence synthesis and recommendation rating.
Methods and Findings
Using a comprehensive checklist for a successful guideline enterprise, we conducted a national modified Delphi consensus process to identify priorities for deprescribing guidelines, then conducted scoping exercises to identify feasible topics, and sequentially developed three deprescribing guidelines. We selected guideline development team members for clinical expertise; a GRADE member worked with staff to ensure guideline development processes were followed. We conducted or used systematic searches and reviews of deprescribing trials of selected drug classes, reviews or systematic reviews of drug class effectiveness, reviews of reviews of drug class harm and narrative syntheses of contextual questions to inform recommendations and guideline development. Our 8 step process for guideline development included defining scope and purpose, developing a logic model to guide the process and generate key clinical questions, setting criteria for admissible evidence and conducting systematic reviews, synthesizing evidence considering additional contextual information and performing quality estimates, formulating recommendations and providing strength estimations, adding clinical considerations, conducting clinical and stakeholder review and finally updating content pre-publication. Innovative aspects of the guideline development process included synthesizing evidence for outcomes of tapering or stopping medication, and incorporating evidence for medication harm into the recommendation strength rating. Through the development of three deprescribing guidelines (for proton pump inhibitors, benzodiazepine receptor agonists and antipsychotics) and associated decision-support algorithms, we were able to gradually hone the methodology; each guideline will be published separately.
Citation: Farrell B, Pottie K, Rojas-Fernandez CH, Bjerre LM, Thompson W, Welch V (2016) Methodology for Developing Deprescribing Guidelines: Using Evidence and GRADE to Guide Recommendations for Deprescribing. PLoS ONE 11(8): e0161248. https://doi.org/10.1371/journal.pone.0161248
Editor: Terence J. Quinn, University of Glasgow, UNITED KINGDOM
Received: March 7, 2016; Accepted: August 2, 2016; Published: August 12, 2016
Copyright: © 2016 Farrell et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Qualitative data are available from the Bruyere Research Ethics Board for researchers who meet the criteria for access to confidential data. Guideline development team members consented to use of their data for developmental evaluation, not for public dissemination.
Funding: The “Deprescribing Guidelines in the Elderly Project” is a research program of The Ontario Pharmacy Research Collaboration (OPEN) which is funded by the Government of Ontario. BF received the funding. The views of the funding bodies have not influenced the content of the guideline; competing interests of each guideline development team member were recorded and addressed. The views expressed in this article are those of the authors and do not represent those of the Government of Ontario. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Grant #06674 http://www.health.gov.on.ca/en/pro/ministry/research/hsrf.aspx.
Competing interests: The authors have declared that no competing interests exist.
Little deprescribing guidance is available to clinicians and the public. Current deprescribing algorithms [1–4] are not class specific and were not developed using a systematic approach. Our guideline development approach addresses these gaps. Comprehensive and explicit identification and evaluation of the literature is needed in the development of evidence-based deprescribing guidelines. Our team developed methods, conducted reviews and implemented three evidence-based deprescribing guidelines in six practice sites. This article describes the methodology utilized by the team, including methods for prioritization, syntheses of evidence and Grading of Recommendations Assessment, Development and Evaluation (GRADE) evidence to recommendation process used for this class–specific evidence-based deprescribing initiative. Examples of content from each deprescribing guideline are included to illustrate how we applied the methods for different topics; readers are referred to the separate guideline publications for comprehensive descriptions.
A companion paper outlines the developmental evaluation method we used to study the development and implementation of the guidelines . Results of the developmental evaluation, including specifics of how the results subsequently affected guideline development processes, will be published separately.
Why are clinical deprescribing guidelines necessary?
Deprescribing is the planned and supervised process of dose reduction or stopping of medication(s) that may be causing harm or are no longer providing benefit. The goal of deprescribing is to reduce medication burden and harm, while maintaining or improving quality of life. This is particularly important to consider as people age, given changes in pharmacokinetics, pharmacodynamics and physiological reserve . Risks of polypharmacy include adverse effects, prescribing cascades and drug interactions, which can lead to morbidity, hospitalization and even death [7–13].
Small studies have demonstrated successful deprescribing approaches,  yet have not been synthesized to produce class-specific deprescribing guidelines. This deficit of guidelines to stop medications stands in contrast to the vast number of guidelines that promote starting medications. Rudimentary guides and generic algorithms to guide deprescribing thought processes exist, and while valuable, they do not explicitly incorporate quantitative evidence for class-specific benefits and harms; it is also unclear how (or if) patient values and preferences contributed to these approaches [1–4]. Prescribers report difficulty in weighing benefits and harms of continuing or stopping medications, as well as pressure to continue to prescribe according to prescribing guidelines . They identify a need to have clear information about benefits and risks of treatment, and a mechanism to elicit patient values and preferences in order to make deprescribing decisions . Patients report the need to understand appropriateness of, and the processes for cessation, in order to feel comfortable with deprescribing . Deprescribing guidelines attempt to address these information needs.
Materials and Methods
We used a comprehensive checklist for a successful guideline enterprise to develop the methods for our deprescribing guidelines.
We began by conducting a national modified Delphi consensus process with potential users of the guidelines (including 11 family physicians, 8 geriatricians, 36 pharmacists and 10 nurse practitioners from 8 Canadian provinces) to identify priorities for deprescribing guideline development . Participants identified the top five topics for evidence-based deprescribing guidelines. We established a Methods Committee, consisting of a physician GRADE methodologist , a pharmacist experienced in conducting systematic reviews and a pharmacist with expertise in managing polypharmacy in geriatrics, to direct and oversee guideline development methods. Core research staff attended a Cochrane Collaboration systematic review workshop, and with Methods Committee support, trained others throughout the project.
We conducted scoping exercises to identify three of the top five priority topics for which deprescribing guideline development would be feasible . These included literature searches conducted in Pubmed, EMBASE and the Cochrane Library to answer the questions (1) what literature exists on the deprescribing of the drug or drug class and (2) what are the benefits and harms of continued use of the drug class. Staff summarized these searches narratively and the Methods Committee, in conjunction with the Deprescribing Guidelines investigator team, decided which drug classes would be feasible candidates for guideline development. These scoping exercises assessed the depth and breadth of literature on the topics (i.e. whether a sufficient body of literature existed from which to develop a guideline), and identified previously conducted systematic reviews. This allowed teams to identify if a de novo systematic review would be required for the guideline.
We established a Guideline Development Team (GDT), including a Methods Committee member for each of three topics–proton pump inhibitors, benzodiazepine receptor agonists and antipsychotics. Each team included a guideline lead, clinical experts (i.e., family physicians, pharmacists, relevant specialists, GRADE methodologist) and a staff coordinator. For example, the BZRA guideline included a family physician (1), pharmacists (2), clinical pharmacologist (1), psychiatrists (2), psychologist (1) geriatrician (1) and GRADE methodologist (1). The names and expertise of each GDT member are described in each guideline publication. Guideline leads had clinical expertise with the topic area and experience conducting systematic reviews for guideline development. We identified clinical experts through the Cochrane network, a network of clinicians interested in the Deprescribing initiatives and through identified experts recommending other experts. Clinical and guideline method expertise contributed to the selection of additional guideline team members. We recorded team members’ conflict of interest declarations (i.e., interests that might impact motivations or decision making because of potential gain–financial, academic advancement, clinical revenue or community standing–or relationships that might affect decision-making).
Each of the three guideline development teams followed the same process (Fig 1), adapted from the AGREE II  and Guidelines 2.0  checklists, focussing on eight key steps (Fig 2). The Methods Committee established processes for communication and task completion and refined these over the sequential development of the three guidelines. Each GDT defined unanimous agreement or majority agreement (>80%) as acceptable for decision-making about the wording of guideline recommendations. Each GDT lead took responsibility for overall inclusion of relevant content and for editing the guideline manuscript to ensure uniform style, format and content throughout.
Step 1: Define scope and purpose of the guideline
Each GDT met either face-to-face and/or by teleconference to determine the target audience, scope and purpose of the guideline. They considered target audiences to be health care professionals involved in the prescribing or monitoring of medication effectiveness and safety within the primary care and long-term care environments. The purpose of each guideline was to provide evidence for the benefits and harms of deprescribing a drug or drug class (including feasibility and safety in terms of not worsening symptoms or causing disease recurrence), evidence of benefit and harm of continuing the drug or drug class, patient preferences and values surrounding deprescribing, resource implications of deprescribing as well as practical guidance on how to implement deprescribing .
Each team identified patients for whom the guideline would (i.e., particular indications) and would not (i.e., those in whom the medication should always be continued or for whom re-evaluation by a specialist would be required prior to deprescribing) apply, as well as other guidelines recommending a limited duration of treatment for certain indications. While our original intention was to develop guidelines to facilitate deprescribing in the elderly, there was such a paucity of trials in this age group that we therefore expanded the inclusion criteria to include all adults (≥18 years).
Step 2: Develop logic model to guide the guideline development process and generate key questions (safety, effectiveness of continuing vs. reducing or discontinuing medications)
The Methods Committee developed a generic logic model to illustrate the possible evidence chain of each guideline (Fig 3). The logic model used the “PICO” framework to identify the Population of interest (adults >18), the Intervention (deprescribing), the Comparator (continuation of the medication) and patient-important Outcomes relevant for decision-making (both positive impact and adverse effects of deprescribing) . Staff, supported by a librarian scientist, conducted preliminary scoping reviews for all types of evidence (ie. systematic reviews, clinical trials, and observational studies) to determine the feasibility of finding evidence to support the evidence chain and to inform each GDT as they worked to define the key questions for each guideline.
Each GDT populated a standard template during their initial meeting, then revised a circulated copy of the key PICO question until consensus was reached. Each team considered and ranked the importance for patients and clinicians on the following outcomes: effect on pill burden, patient satisfaction, medication prescription numbers, cost/resource implications, adherence, health care utilization and stabilization, worsening or improvement of a symptom or disease, quality of life and changes in physical or cognitive function.
Driven by their clinical experience, each GDT also identified additional contextual questions, for example: What are alternative, safer, effective pharmacological or non-pharmacological options for treatment of the symptom or condition? Are there certain populations that might be more amenable to deprescribing? What tapering strategies and monitoring approaches should be used to safely deprescribe the medication?
Step 3: Agree on criteria for admissible evidence; use or conduct systematic review
An expert medical librarian and a systematic review methodologist with experience conducting Cochrane reviews developed the search strategies to assess benefit or harms of deprescribing the drug class (see sample search strategy—S1 Appendix). In situations where a published systematic review of deprescribing met the criteria, steps were taken to ensure it was up-to-date for use in the guideline. If there were no deprescribing trials for a particular indication, the focus of the de novo systematic review shifted to one studying effectiveness of initiating the drug class for that indication, recognizing that the time-horizon and effect-sizes are likely to be different for effects in new users as compared to discontinuation effect in prevalent users. Staff reviewed the search strategy results and tested them with known articles. Staff, medical and pharmacy students, and pharmacy residents (minimum of two staff/students per systematic review) completed the systematic reviews. Each team registered its systematic review titles and protocols with the Cochrane Collaboration, or Prospero [25,26]. They used the Cochrane methodological expectations for conduct of intervention reviews and the Cochrane handbook and reported results of reviews using the PRISMA (Preferred Reporting in Systematic Reviews and Meta-Analyses) reporting guidelines with a PRISMA flow chart.
Step 4: Synthesize the evidence, assess quality of studies; consider additional information (e.g. benefits, harms, values/preferences, cost and resource implications, other guidelines)
Each GDT used the GRADE approach to provide a formal rating of the synthesized evidence from the systematic review for the selected critical and important outcomes of deprescribing  (e.g. sleep quality for benzodiazepine receptor agonists). A summary of findings table of the deprescribing systematic review results was developed (see sample summary of findings table–S2 Appendix). In situations where a systematic review of initiating the drug class was conducted, a summary of findings table was produced and effect sizes considered in weighing the balance of continuing the medication in light of other information such as harms data, patient preference and resource implication data.
The librarian conducted a literature search to facilitate an English only review of reviews of harms using the methods outlined by Smith et al.  Relevant literature from the search results was independently identified by two research assistants using the following inclusion criteria: systematic reviews of randomized controlled trials (RCTs) or observational studies with a priori objectives of assessing harms attributable to the drug(s) of interest. The research assistants met with one investigator to reconcile the resulting list of studies and resolve any disagreements regarding selection of relevant studies.
The librarian conducted additional literature searches to provide relevant content for considering other contextual questions (patient/family values/preferences, costs and resource implications, and other guidelines, including benefits of treatment). An example search strategy for values and preferences is in S3 Appendix. Each GDT member took responsibility for reviewing and summarizing literature specific to at least one of these topics.
Step 5: Formulate recommendations and assess strength of recommendations
All evidence (summary of findings, quality of evidence, harm, patient preferences and values, resource implications) was synthesized into recommendations using the GRADE Evidence to Recommendations Table (see sample in S4 Appendix).  Judgements of directness (important differences between the PICO elements being considered for our guideline, and the PICO elements from studies we synthesized)  were made on the basis of clinician expertise and judgement, as well as by assessing elderly-specific literature (if available). Each GDT lead presented their evidence and draft clinical actions to their team members and provided GDT members with the Evidence to Recommendations Table. During this stage, the GDT discussed decisions regarding the quality of evidence and strength of recommendations. Finally, each GDT coordinator conducted voting on exact recommendation wording via email; recommendations were revised if necessary and revised versions presented to the GDT for approval. GDT members identified gaps in evidence for inclusion in the guideline.
Step 6: Add clinical considerations
Each GDT generated a series of questions that clinicians typically faced when trying to decide when and how to stop a medication and what to monitor for the effects of discontinuing the medications. Members considered relevant information from the evidence reviews (such as how the medication class was tapered, over what period of time and what monitoring parameters were observed during the trials) and their own clinical experience to provide guidance on these questions. Sample questions are outlined in Fig 4.
Step 7: Conduct review and piloting: clinical review and stakeholder review using AGREE II
A practicing family physician, nurse practitioner and pharmacist (who were not members of the Deprescribing project) provided a clinical review of each guideline using the AGREE II Global Rating Scale  to guide their evaluation in rating the scope and purpose of the guideline, stakeholder involvement, rigour of development, clarity of presentation, applicability and editorial independence . The GDT subsequently made revisions to guideline content in order to improve understanding and facilitate implementation. Staff next developed a PowerPoint presentation and one-page algorithm as a decision support tool for each guideline. The GDT lead and coordinator presented both in six practice sites (three Long-Term Care and three Family Health Teams) in an urban community setting in Ottawa, Ontario. The research team gathered feedback from physicians, pharmacists, nurses and site administrators at the initial introduction of each guideline and after approximately four months of use . Site representatives were asked about patient feedback and this was taken into account for revisions and development of implementation tools. Concurrently, relevant stakeholders (physician, pharmacist and nursing groups, whose reviewers were not members of the guideline development teams) for each guideline reviewed the guideline using AGREE II Global Rating Scale and/or their internal guideline review processes, provided feedback and considered endorsing the guideline. For example, we approached the College of Family Physicians of Canada, the Canadian Pharmacists Association, the Canadian Nurses Association and organizations specific to each guideline topic; more details are provided in each guideline paper.
Step 8: Update recommendations and evidence pre-publication
Prior to publication, the GDT incorporated feedback from stakeholders and pilot sites, and updated guideline sections if new relevant evidence was available.
Dissemination of each guideline is planned via open-access publication and use of a website (deprescribing.org) to house implementation tools (e.g. PowerPoints, algorithms). Knowledge translation will be facilitated through social media (Facebook, Twitter @deprescribing) and conference presentations/workshops.
This 8 step systematic process helped interdisciplinary teams conceptualize the potential benefits and harms of deprescribing and to consistently evaluate the evidence for each of the three medication classes. We believe these are the first deprescribing guidelines to have been developed to conform with AGREE II, the current quality standard for guideline development, and applying the GRADE recommendation process. In keeping with these standards, guideline development teams paid careful attention to defining scope and purpose, explicitly identifying key questions and important outcomes, ensuring stakeholder involvement (through both external groups and pilot test sites), using rigorous search and synthesis processes, ensuring clarity of presentation, assessing applicability (through pilot testing in both community and long-term care settings), documenting competing interests and maintaining editorial independence.
Consistent with research that demonstrates guidelines are more likely to be implemented when accompanied by guideline implementation tools [34,35], each GDT developed PowerPoint presentations for communicating key guideline aspects to users, as well as decision-support tools in the form of algorithms. Over time, with generation and piloting of each subsequent guideline, development of the algorithm early on in the teams’ processes ensured that end-user feedback was taken into account and that guidelines addressed key decision points in the algorithms that users felt needed to be included. This is consistent with the findings of Gagliardi et al, whose qualitative research examining guideline implementation tools also found that planning for them needed to commence in conjunction with guideline development but was often complicated by lack of resources .
Strengths of our process included engagement of clinicians in a feedback process using AGREE II, inclusion of all GDT members with an active role in writing sections of the guideline, using the GRADE process to make judgements about directness and values transparent, honing a streamlined process for guideline development and incorporating the development of guideline implementation tools. The research team set out to develop and pilot three guidelines over 30 months. We conducted a developmental evaluation strategy of both the guideline development and implementation processes . Results of this analysis will be published separately.
Limitations encountered in our deprescribing guideline development process included the paucity of RCT evidence for deprescribing; this narrowed the choice of topics for guideline development using the GRADE method and highlights an important gap in research that funding agencies should note. Guideline development team members relied on observational evidence for harms data, surveys and interviews for patient preferences data, and expert clinical opinion for clinical considerations pieces when other evidence was unavailable. Our intention was to incorporate frontline patient perspective through interviews with patients in the pilot sites (actual users of the guidelines), however, we were unable to recruit sufficient numbers of patients to complete a valid qualitative analysis. Instead, we subsequently incorporated feedback from clinicians who had implemented the guidelines with patients, relying on their perspective to provide proxy patient input. We recognize the limitation with this approach and recommend going forward, that deprescribing guideline developers include representative patient users on guideline development panels from the outset. We recognize this approach is also fraught with challenges  but feel strongly that such engagement would improve both the clarity and communication of recommendations and design of relevant implementation tools. Finally, research questions for which there was often little, if any, evidence included whether a drug was known to have benefits or cause adverse effects in the elderly and, in terms of the latter, what impact these might have (e.g. emergency room visits, hospitalization), whether a drug should be tapered or simply stopped, what tapering regimens were most effective and what parameters should be monitored, how often and for how long.
Evidence-based deprescribing involves systematic search, evaluation and synthesis of evidence on benefits and harms to formulate recommendation on tapering or stopping medications. Decisions around deprescribing can be clinically difficult to make. We have developed methodology to create guidelines that synthesize the information needed to support deprescribing decisions. This methodology could be used as a template for future evidence based deprescribing guidelines.
Our guidelines take into account evidence of benefit of the medication class, estimations of their harm and evidence of the feasibility and outcomes of deprescribing–the latter two topics representing key innovations in guideline development. The GRADE approach assisted in rating the quality of evidence, particularly for the elderly population subgroup where possible, as well as taking into account patient preferences and resource implications when formulating recommendations. The process fulfills the AGREE II criteria, the current quality standard for guidelines development.
S1 Appendix. Sample Medline search strategy for key PICO question of PPI deprescribing guideline.
S2 Appendix. Sample summary of findings table from antipsychotic deprescribing guideline.
S3 Appendix. Sample search strategy for contextual question on values and preferences of antipsychotic use in dementia.
S4 Appendix. Example of evidence to recommendations table from BZRA deprescribing guideline.
We sincerely thank Lisa Pizzola and Matthew Hogel, who, as guideline coordinators, contributed helpful input to improve guideline development processes.
- Conceptualization: BF KP CRF.
- Funding acquisition: BF.
- Methodology: BF KP CRF VW LB WT.
- Project administration: BF.
- Supervision: BF KP.
- Visualization: BF KP WT.
- Writing - original draft: BF KP CRF WT.
- Writing - review & editing: BF KP WT CRF LB VW.
- 1. Hardy JE, Hilmer SN. Deprescribing in the Last Year of Life. 2011;41(2):146–51.
- 2. Garfinkel D, Mangin D. Feasibility study of a systematic approach for discontinuation of multiple medications in older adults: addressing polypharmacy. Arch Intern Med. 2010;170(18):1648–54. pmid:20937924
- 3. Scott IA, Hilmer SN, Reeve E, Potter K, Le Couteur D, Rigby D, et al. Reducing inappropriate polypharmacy: the process of deprescribing. JAMA Intern Med. 2015;175(5):827–34. pmid:25798731
- 4. Lemay G, Dalziel B. Better prescribing in the elderly. 2012;2(3):20–6.
- 5. Conklin J, Farrell B, Ward N, McCarthy L, Irving H, Raman-Wilms L. Developmental evaluation as a strategy to enhance the uptake and use of deprescribing guidelines: protocol for a multiple case study. Implement Sci. 2015 Jun;10(1):91.
- 6. Sera LC, McPherson ML. Pharmacokinetics and pharmacodynamic changes associated with aging and implications for drug therapy. Clin Geriatr Med. 2012;28(2):273–86. pmid:22500543
- 7. Reason B, Terner M, Moses McKeag A, Tipper B, Webster G. The impact of polypharmacy on the health of Canadian seniors. Fam Pract. 2012;29(4):427–32. pmid:22223743
- 8. Shah BM, Hajjar ER. Polypharmacy, adverse drug reactions, and geriatric syndromes. Clin Geriatr Med. 2012;28(2):173–86. pmid:22500537
- 9. American Geriatrics Society 2015 Updated Beers Criteria for potentially inappropriate medication use in older adults. J Am Geriatr Soc. 2015;63(11):2227–2246. pmid:26446832
- 10. Hajjar ER, Cafiero AC, Hanlon JT. Polypharmacy in elderly patients. Am J Geriatr Pharmacother. 2007;5(4):345–51. pmid:18179993
- 11. Wu C, Bell CM, Wodchis WP. Incidence and economic burden of adverse drug reactions among elderly patients in Ontario emergency departments: a retrospective study. Drug Saf. 2012;35(9):769–81. pmid:22823502
- 12. Jyrkkä J, Enlund H, Korhonen MJ, Sulkava R, Hartikainen S. Polypharmacy Status as an Indicator of Mortality in an Elderly Population. Drugs Aging. 2009;26(12):1039–48. pmid:19929031
- 13. Budnitz DS, Lovegrove MC, Shehab N, Richards CL. Emergency Hospitalizations for Adverse Drug Events in Older Americans. N Engl J Med. 2011;2002–12. pmid:22111719
- 14. Iyer S, Naganathan V, Mclachlan AJ, Couteur DG Le. Medication Withdrawal Trials in People Aged 65 Years and Older A Systematic Review. 2008;25(12):1021–31. pmid:19021301
- 15. Anderson K, Stowasser D, Freeman C, Scott I. Prescriber barriers and enablers to minimising potentially inappropriate medications in adults: a systematic review and thematic synthesis. BMJ Open. 2014;4(12):e006544. pmid:25488097
- 16. Schuling J, Gebben H, Veehof LJG, Haaijer-Ruskamp FM. Deprescribing medication in very elderly patients with multimorbidity: the view of Dutch GPs. A qualitative study. BMC Fam Pract. 2012;13(1):56.
- 17. Reeve E, To J, Hendrix I, Shakib S, Roberts MS, Wiese MD. Patient Barriers to and Enablers of Deprescribing: a Systematic Review. Drugs Aging. 2013;30(10):793–807. pmid:23912674
- 18. Schünemann HJ, Wiercioch W, Etxeandia I, Falavigna M, Santesso N, Mustafa R, et al. Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise. CMAJ. 2014;186(3):E123–42. pmid:24344144
- 19. Farrell B, Tsang C, Raman-Wilms L, Irving H, Conklin J, Pottie K. What are priorities for deprescribing for elderly patients? Capturing the voice of practitioners: a modified delphi process. PLoS One. 2015;10(4):e0122246. pmid:25849568
- 20. Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011;64(4):383–94. pmid:21195583
- 21. Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.
- 22. Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, et al. AGREE II: advancing guideline development, reporting and evaluation in health care. CMAJ. 2010;182(18):E839–42. pmid:20603348
- 23. Connor Gorber S, Singh H, Pottie K, Jaramillo A, Tonelli M. Process for guideline development by the reconstituted Canadian Task Force on Preventive Health Care. CMAJ. Canadian Medical Association; 2012;184(14):1575–81. pmid:22891209
- 24. Guyatt GH, Oxman AD, Kunz R, Atkins D, Brozek J, Vist G, et al. GRADE guidelines: 2. Framing the question and deciding on important outcomes. J Clin Epidemiol. 2011;395–400. pmid:21194891
- 25. Boghossian T, Rashid J, Welch V. Deprescribing versus continuation of chronic proton pump inhibitor use in adults (Protocol). Cochrane Libr. 2015;(11):1–28.
- 26. Thompson W, Quay T, Tang J, Bjerre L, Farrell B. Atypical antipsychotics for insomnia: a systematic review and meta-analysis protocol. PROSPERO. 2015;CRD4201501.
Higgins J, Green S. Cochrane handbook for systematic Reviews of Interventions. 2011.
- 28. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JPA, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 2009;339:b2700–b2702. pmid:19622552
- 29. Balshem H, Helfand M, Schünemann HJ, Oxman AD, Kunz R, Brozek J, et al. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol;2011: 401–6. pmid:21208779
- 30. Smith V, Devane D, Begley CM, Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. BMC Med Res Methodol. 2011;11(1):15. pmid:21291558
- 31. Andrews JC, Schünemann HJ, Oxman AD, Pottie K, Meerpohl JJ, Coello PA, et al. GRADE guidelines: 15. Going from evidence to recommendation-determinants of a recommendation’s direction and strength. J Clin Epidemiol. 2013;66(7):726–35. pmid:23570745
- 32. Guyatt GH, Oxman AD, Kunz R, Woodcock J, Brozek J, Helfand M, et al. GRADE guidelines: 8. Rating the quality of evidence indirectness. J Clin Epidemiol. 2011;64(12):1303–10. pmid:21802903
- 33. Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, et al. AGREE II: advancing guideline development, reporting and evaluation in health care.J Clin Epidemiol. 2010;63(12):1308–11. pmid:20656455
- 34. Pronovost PJ. Enhancing Physicians’ Use of Clinical Guidelines. JAMA. 2013;310(23):2501–01. pmid:24310916
- 35. Greenhalgh T, Howick J, Maskrey N. Evidence based medicine: a movement in crisis?. BMJ. 2014;348.
- 36. Gagliardi AR, Brouwers MC, Bhattacharyya OK. The development of guideline implementation tools: a qualitative study. CMAJ Open. 2015;3(1):E127–33. pmid:25844365
- 37. Boivin A, Green J, van der Meulen J, Légaré F, Nolte E. Why consider patients’ preferences? A discourse analysis of clinical practice guideline developers. Med Care. 2009;47(8):908–15. pmid:19543120