Surgical site infections (SSI) occur in up to 10% of surgeries. Wound care practices to prevent infections are guided by Clinical Practice Guidelines (CPGs), yet their contribution to improving patient outcomes relies on their quality and adoption in practice. We critically evaluated the quality of CPGs for SSI prevention during pre-, intra- and post-operative phases of care.
We systematically reviewed the literature from 1990–2018 using the Cochrane Library, CINAHL, EMBASE, MEDLINE, ProQuest databases and five guidelines repositories. We extracted characteristics of each guideline using purposely-developed data collection tools. We assessed overall quality using the Appraisal of Guidelines for Research and Evaluation II (AGREE II) tool.
Combined searches of databases and repositories yielded 5,910 citations. Of these, we reviewed 215 full text documents. The final sample included 15 documents: 6 complete CPGs, 3 CPG updates, and 6 supplementary documents. The overall %mean scores across AGREE II domains for CPGs were: 1) scope and purpose (%mean ± SD = 86.3±23.5); 2) stakeholder involvement (%mean ± SD = 64±31.0); 3) rigour of development (%mean ± SD = 68.7±30.6); 4) clarity and presentation (%mean ± SD = 88.5±16.7); 5) applicability (%mean ± SD = 44±30.2); and, 5) editorial independence (%mean ± SD = 61±37.6). Based on individual AGREE II domains and overall scores, we appraised 4 out of 6 CPGs (inclusive of updates) as “recommended” for use in practice. Overall agreement among appraisers was excellent (ICC 0.86 [95%CI 0.73–0.94] - 0.98 [95%CI 0.96–0.99]; p <0.001).
International interest in CPG development has resulted in refinements to methodologies, which has led to improvements in the overall quality of the product.
Citation: Gillespie BM, Bull C, Walker R, Lin F, Roberts S, Chaboyer W (2018) Quality appraisal of clinical guidelines for surgical site infection prevention: A systematic review. PLoS ONE 13(9): e0203354. https://doi.org/10.1371/journal.pone.0203354
Editor: Duncan Chambers, University of Sheffield, UNITED KINGDOM
Received: March 8, 2018; Accepted: August 20, 2018; Published: September 13, 2018
Copyright: © 2018 Gillespie et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: This study was supported by a Gold Coast Health and the Gold Coast Hospital Foundation Research Grant which was obtained by BMG (grant number: RGS2017-LG0007; Gold Coast University Hospital url: https://www.goldcoast.health.qld.gov.au/hospitals-and-centres/gold-coast-university-hospital). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Over 234 million surgeries are performed around the world every year, yet despite the remarkable advances in surgical technologies and anaesthetic techniques, surgical site infections (SSIs) remain a major cause of patient morbidity and mortality.[2, 3] SSIs are potential complications associated with any surgical procedure; however they are the most preventable hospital acquired infection (HAI). It is estimated that SSI will occur in up to 9.5% of inpatient surgical procedures.[3, 5] SSI is defined as any infection occurring within 30 days after surgery or within 12 months of surgical implantation of a prosthesis or foreign body. Evidence-based wound care plays a significant role in reducing the physical, psychological, social and economic burden SSIs have on healthcare systems, patients and their families.
Decisions that health professionals make relative to wound management have important implications for patient outcomes. Clinical Practice Guidelines (CPGs) offer guidance in the standardisation of care, and improve the allocation and utilisation of finite healthcare resources and reduce waste. However, the potential of CPGs to enhance wound care practice and reduce the risk of SSI is dependent on their quality, as well as uptake and adoption in practice . The purpose of this systematic review was to evaluate the quality of the CPGs and strength of their recommendations that inform wound care practices in SSI prevention. Identification of the strengths and limitations of the CPGs may ultimately drive future improvements in their quality and applicability. The results of this review may also assist in the decision making of policy makers and senior clinicians relative to implementing evidence informed practices in SSI prevention.
We were guided by the Preferred Reporting Items for Systematic Reviews and Meta‑Analysis (PRISMA) statement , PRISMA 2009 Checklist (S1 Checklist), and the Cochrane Handbook for Systematic Reviews of Interventions to undertake this systematic review and report the results. As part of this process, we identified research questions a priori and registered the review protocol with the international prospective register of systematic reviews (PROSPERO registration number: 42017073205).
The objective of this systematic review was to critically appraise the overall quality of published guidelines for the prevention of SSI using the Appraisal of Guidelines for Research and Evaluation II (AGREE II) tool. Subsumed under this objective were the following questions:
- Using the AGREE II tool, what is the quality of the CPGs and strength of their recommendations for the prevention of SSI?
- Have the CPGs been revised, updated or improved over time?
As part of the appraisal process, we evaluated the similarities and differences of the main recommendations of the guidelines in parallel with the evaluation of their quality.
We applied the following inclusion criteria:
- Published international and national guidelines on the management and/or prevention of SSI;
- Published as full-text between January 1990 to February 2018;
- Guidelines published in English, as these are the most accessible and widely available;
- Most recent complete guideline (from a single working group i.e. CDC) and any partial revisions for the guideline published thereafter;
- SSI prevention/management guidelines that make recommendations across the pre-operative, intra-operative and post-operative phases; and,
- Include an explicit statement identifying the document as a ‘guideline’.
We applied the following exclusion criteria:
- Guidelines under development;
- Guidelines specific to one institution or surgical specialty (i.e. local hospital guidelines or orthopaedic surgery, e.g., Smith & Dahlen);
- Guidelines inclusive of only one phase of care, e.g., Ubbink et al (i.e., postoperative phase focusing on wound care and pain management)
- Complete guidelines with publication dates that have been superseded by more recent complete guidelines (i.e. the 1999 CDC guidelines have superseded the 1992 guidelines);
- Guidelines that only cover one aspect of SSI prevention (i.e. antibiotic prophylaxis); and,
- Clinical practice standards, defined as a statement reached through consensus, which clearly identifies the desired outcome. Usually used in audit as a measure of success.
Data sources and search strategy
With the assistance of a health librarian, we conducted systematic electronic searches using the Cochrane Library, Cumulative Index of Nursing and Allied Health Literature (CINAHL), EMBASE, MEDLINE and ProQuest databases. We tailored searches to the capabilities of each database, and where appropriate used either Medical Subject Headings (MeSH) or keywords, and Boolean connectors AND, OR and NOT to combine search terms. Since computerised searches identify only 50% of eligible studies, ancestry searching and journal hand-searching were also conducted. We imported all database results into an Endnote (v X7, Clarivate Analytics) reference manager program prior to screening. We supplemented the database searches by searching guidelines repositories. These included The National Guideline Clearinghouse, New Zealand Guidelines Group, The National Institute for Health and Care Excellence (NICE), The National Health and Medical Research Council (NHMRC)–Australian Clinical Practice Guidelines, CPG Infobase: Clinical Practice Guidelines (Canadian Medical Association), Scottish Intercollegiate Guideline Network (SIGN), Clinical Key (Elsevier), and BMJ Best Practice. An example of the searches undertaken across database and guideline repositories is provided in S1 Table, MEDLINE (EBSCO) search strategy and guideline repository searches.
Data screening and extraction
Groups of titles and abstracts were assigned to three review authors who independently screened their allocated sample, and those deemed to meet the inclusion criteria were assessed further in full text. A fourth reviewer arbitrated where there was a lack of clarity around inclusion. We documented the reasons for exclusion. One review author performed data extraction using an extraction tool specifically developed for this review based on guideline characteristics relative to quality (AGREE II ).
Quality assessment of CPGs
To appraise the overall quality of each included CPG, we used the Appraisal of Guidelines, Research and Evaluation (AGREE) II statement  to guide our systematic assessment of all published CPGs eligible for inclusion. It is generic and can be applied to guidelines in any disease area targeting any step along the healthcare continuum, including screening, diagnosis, treatment or interventions. The AGREE II tool has 23 items comprising six quality-related domains (Scope and purpose, Stakeholder involvement, Rigour of development, Clarity of presentation, Applicability and Editorial independence) and a 24th overall assessment item. A brief description of the criteria in each of these domains is included as an S2 Table. AGREE II domain definitions. The AGREE tool was updated to AGREE II to (i) encompass a greater consideration of health generally as opposed to a specific ‘clinical’ focus; (ii) move items to more appropriate domain categories; and (iii) the addition of items (i.e. strength and limitations of the body of evidence in Domain 3: Rigour of Development).
The redesigned AGREE II tool uses a 7-point Likert scale, 1 = strongly disagree through to 7 = strongly agree. Scores for each domain were calculated by summing all scores of the individualised items within each domain and then standardising as follows: (obtained score minus minimal possible score) divided by (maximum possible score minus minimum possible score). The minimum standardised score for each domain was 0% and the maximum was 100%. Scores were converted to percentages values for %mean score determination. The scale was transformed to the following: 1 = 14.29%, 2 = 28.57%, 3 = 42.86%, 4 = 57.14%, 5 = 71.43%, 6 = 85.71%, and 7 = 100%. A guideline is ‘‘ recommended” if most of the domains (≥ 4) scored above 50%. A guideline is ‘‘recommended with modifications” if 3 domain items scored above 50%. A guideline is ‘‘not recommended” if ≥4 domains scoring less than 50%. Appraisers assigned scores depending on the completeness and quality of reporting and scores increased as more criteria were met.
CPGs deemed eligible for inclusion were appraised independently by members of the authorial team using the AGREE II tool. Each CPG was assigned three appraisers. The AGREE II requires at least two appraisers to reach acceptable interrater agreement on the tool. We classified the degree of agreement according to the scale by Landis and Koch; poor (0.00), slight (0.00 and 0.20), fair (0.21 to 0.40), moderate (0.41 to 0.60), substantial (from 0.61 to 0.80), and very good or almost perfect (0.81 to 1.00). Prior to completing appraisals of each CPG, all review authors completed the AGREE II Online Training Tool  and participated in three rounds of calibration. Authors completed online appraisals using the My AGREE PLUS platform. During the quality appraisal process, we met regularly to discuss results, clarify information and resolve differences by consensus. We measured the mean proportion of agreement relative to overall quality (AGREE II) of included CPGs among three assessors in each AGREE domain using the ICC with 95% confidence intervals (CI).
Our electronic database searches retrieved 5,630 documents; of these, we considered 212 for full text screening. Among these, we excluded 206 documents. Using guidelines repositories, we identified 280 documents, and of these, excluded 276 documents. In total, we included 15 documents in the final analysis. The total number of guidelines included six complete CPGs,[5, 18–22] 3 updates [18, 23–25] and six additional documents (including a supplement)[25–29] that supplied information not present in the original or update CPG documents relevant for appraisal. Figs 1 and 2 illustrate the complete selection process based on the database and guidelines repositories using PRISMA  flow charts. Table 1 details the characteristics of the six CPGs and their included updates. The included CPGs and their updates were published between 1999 and 2018. Of the six complete CPGs, 3 [5, 18, 19] were developed in the United States. Refer to S3 Table–Summary of sources where CPGs were obtained.
Three assessors appraised each CPG. The %mean scores for each AGREE II domain were calculated by summing the scores of the individual items in domains and then standardising minimum/maximum scores ranging from 0% to 100%. The results were as follows: 1) scope and purpose (%mean ± SD = 86.3±23.5); 2) stakeholder involvement (%mean ± SD = 64±31.0); 3) rigour of development (%mean ± SD = 68.7±30.6); 4) clarity and presentation (%mean ± SD = 88.5±16.7); 5) applicability (%mean ± SD = 44±30.2); and, 5) editorial independence (%mean ± SD = 61±37.6).
The overall quality scores of each guideline (including updates) across each domain of the AGREE II are presented in Table 2. Where CPGs included updates, we appraised the update as part of the original guideline. In the first AGREE II domain, Scope and Purpose, quality scores ranging from 39% (±14.3) to 100% (±0) with 5 of 6 CPGs scores over 50%. The second domain Stakeholder Involvement, mean scores across CPGs varied from 15% (±13.3) to 100% (±0), with 4 of 6 of CPGs with scores greater than 50%. In the third AGREE II domain, Rigour of Development, mean scores ranging from 21% (±20.5) to 97% (±6.9) with 4 of 6 CPGs scored over 50%. The fourth domain, Clarity of Presentation mean scores varied from 56% (±21.6) to 100% (±0) and all CPGs scored over 50%. In the fifth domain, Applicability, mean scores were much lower overall, ranging from 4% (±6.5) to 86% (±13.4), and only 3 of 6 CPGs had scores greater than 50%. In the sixth domain, Editorial Independence, 4 of 6 CPGs scored over 50%, with scores ranging from 11% (±11.7) to 100% (±0). Based on the appraisal of individual AGREE II domains and overall scores, 4 of 6 of the included CPGs were rated as “recommended”.
Across the AGREE II domains, ICC coefficients among appraisers ranged from 0.86 (95%CI 0.73–0.94) to 0.98 (95%CI 0.96–0.99; p<0.001), indicating an almost perfect level of agreement.
Levels of evidence used in CPGs to inform recommendations
Table 3 shows the levels of evidence for recommendations across each of the three phases of surgical care, i.e., pre-operative, intra-operative and post-operative. Two of the recently developed complete guideline,[21, 22] and recent updates of two others [23, 30] used the GRADE system to rank recommendations. Only one CPG  was developed using a working group (based on expert opinion). Comparatively, there was consistent agreement in the ranking of recommendations across CPGs (including updates) relative to the following SSI prevention interventions; hair removal, antibiotic prophylaxis, and the wearing of surgical attire. However, across most other SSI prevention interventions/strategies in the preoperative and intraoperative period, agreement relative to level of evidence and the number of recommendations was inconsistent. S4 Table–Evidence level systems used across CPGs details the different evidence systems used for each recommendation identified in each CPG. S5 Table–Recommendations across all CPGs that informed Table 3 details specific recommendations across each CPG. The 2017 CDC guideline  included content specific to prosthetic joint arthroplasty. There were far fewer recommendations identified across included CPGs in relation to the post-operative phase, particularly pertaining to wound care.
This is the first systematic evaluation of the quality of SSI prevention guidelines to our knowledge. Generally, the quality of these guidelines was acceptable with most evaluated as “recommended”. A “good” guideline should be scientifically valid, practical, consistent and should ultimately improve the outcomes of patients. Of the CPGs identified in 235 studies assessing the effectiveness and efficiency of dissemination and implementation strategies, only 3% of the guidelines were based on good evidence. Our review identified an overall improvement in the quality of the SSI prevention guidelines over time, albeit that some of the main recommendations are based on weak/low grade or inconclusive evidence. Up to 30% of all medical care adds no value to patients, and may in fact lead to harm. Despite this, there are many interventions that are based on questionable evidence, and their inclusion in CPGs has been labelled an “illusionary attempt to embrace the entire clinical reality.” Clearly, including small trials reporting weak evidence in CPGs have a bearing on the quality and strength of the recommendations identified.
Recommendations across the reviewed CPGs were reasonably consistent for three SSI prevention strategies but developers used different classification systems to indicate the levels of evidence across the studied guidelines. Of concern is the number of unresolved issues across the reviewed CPGs, which demonstrates substantial gaps in the research evidence base. Notably, very few recommendations were identified in relation to wound care strategies, which is indicative of the paucity of robust evidence. This partly explains the undesirable practice variation in wound care.[10, 36] Some experts  have warned against the increase in the use of low grade recommendations for which the evidence is inconclusive or weak. Choosing Wisely campaigns attempt to address this through specialty-specific lists of recommendations of ‘things that clinicians and patients should question’.
Overall, %mean scores were higher in 4/6 domains: scope and purpose, rigour of development, clarity of presentation, and editorial independence. Our results are similar to other CPG reviews covering different clinical topics.[31, 39, 40] These results may reflect ongoing improvements in CPG methodology, which have advanced over the past decade. As the methodology for developing guidelines becomes more established, rigorous and accepted internationally, it will become more readily adopted. Consequently, the criteria used to assess CPGs—based on the methodology—will also improve.
In relation to stakeholder involvement, most included CPGs described the representation of various health professional groups. The inclusion of experts from different professional disciplines acknowledges the importance of a multidisciplinary and collaborative approach that is needed to implement interventions in the prevention of SSI. Still, only two CPGs included patients and their representatives (i.e., or parents/guardians as representatives of patients’ welfare) in guideline development. One of the pillars of evidence-based medicine is patient-centeredness, which is manifest in care that is respectful of and responsive to the expectations, preferences and experiences of patients.[32, 42, 43] Ultimately, patient values and preferences should, where possible, inform clinical decisions. The increasing uptake of Patient and Public Involvement (PPI) groups encourages research development processes to include patients so that the research is ‘by’ them and not just ‘about’ them[44, 45]. As such, guideline developers should consider integrating healthcare consumers in future CPG updates to make them even more comprehensive and relevant.
Applicability is critical in the implementation of a guideline. In our review, this domain scored much lower than the other five domains. A recent systematic review  of 20 studies including 137 guidelines that used AGREE to assess CPGs from 2008–13 found that applicability scored lower than all other domains, and did not significantly improve over time. It is important that guidelines can be adapted to suit different clinical and financial contexts. Clearly, the local context profoundly influences applicability, and will therefore have a significant impact on adoption of the guideline. For instance, in SSI prevention it would be meaningless to recommend a practice or an intervention (e.g., use of pre-warming device, hair removal), if the device is not available or the practice contravenes the cultural norms of a specific setting/group. Nonetheless, this does not negate the imperative for having guidelines based on best evidence. Rather, it emphasises the extent to which evidence obtained in a specific setting is generally valid or applicable to other contexts or situations. While the AGREE II tool is designed to evaluate the overall methodological quality of CPGs, the Appraisal of Guidelines Research and Evaluation-Recommendations Excellence (AGREE-REX)  has been recently developed to supplement this tool. The AGREE-REX specifically evaluates guideline recommendations relative to trustworthiness, suitability and feasibility of implementation in a particular context. The AGREE-REX tool is still under going further refinement , but goes some way to addressing issues related to the local context.
In the reviewed CPGs, there was limited consideration of resource implications. Conversely, with few health economic studies undertaken to evaluate strategies in SSI prevention, guideline developers are challenged to include evidence on the economic benefit of interventions in this field. Economic evaluations have the potential to provide evidence for what works best and for what works most efficiently in real-world practice settings, to ultimately inform healthcare decision-making. Clearly, information deficiencies drive waste, and can lead to the overuse of interventions and treatments that are of little, if any value or benefit to patients.[38, 50]
Strengths and limitations
As with all systematic reviews, we acknowledge some limitations. The inclusion of CPGs covering all phases of surgical care and across all specialties meant that we necessarily excluded high quality CPGs  that were more focussed and specific, and perhaps more user-friendly to busy clinicians, such as Ubbink et al. Although our search methods were exhaustive and robust, we may have possibly missed other CPGs and updates. However, our extensive search strategy covered all indexed and grey literature, and used multiple appraisers who undertook training and calibration to assess the quality of the CPGs. We used an appraisal tool with established validity and reliability  and all reviewers independently appraised CPGs. However, there may be different levels of understanding of the AGREE II tool among appraisers. To address this, we held regular meetings to ensure consistency in the appraisal process across the included CPGs. These discussions offered appraisers the opportunity to present information overlooked by others in the team, therefore clarifying and increasing understanding of the criteria upon which to evaluate the CPGs. Finally, our research team comprised of healthcare professionals from varied professional disciplines, research expertise and experience, thus adding a deeper dimension to guideline interpretation and appraisal.
Implications for translation
Implementation of evidence-based information remains a challenge in many healthcare contexts and it is often difficult to assess application and performance of a CPG in clinical practice. It takes approximately 5 years for any given CPG to be adopted into routine clinical practice and even the broadly accepted guidelines are often not fully followed.[32, 42] Multiple factors influence guideline use including patient, provider, institutional context and systems issues; yet implementation is meant to overcome these barriers.[32, 46] Implementation tools that increase guideline accessibility using a variety of user-friendly formats. For instance, presenting information found in the CPGs as recommendations with evidence summaries, repositories for tools for implementation, and implementation plans and toolkits make guidelines more accessible. Developers should also consider the feasibility and acceptability of implementing SSI prevention interventions across patient groups and different clinical settings, including those in developing countries.
Good information is essential for choosing wisely clinical interventions and treatments in SSI prevention, thus avoiding wasteful healthcare. Most of the reviewed CPGs lacked information about cost-effectiveness and risk-benefit analyses of the strategies used in SSI prevention. Many of the SSI prevention interventions used across the pre-, intra- and post-operative periods have never been subjected to rigorous economic evaluation, but nevertheless continue to be used in clinical practice. Thus, it is important to conduct rigorous parallel economic evaluations alongside trials of clinical effectiveness, as this will provide greater guidance to healthcare decision and policy makers. In terms of applicability and implementation, the inclusion of cost analyses studies in CPGs will assist clinicians in selecting the best available evidence-based options in healthcare organisations with limited resources.[49, 51]
Successful uptake of CPGs depends on clinicians and decision makers trusting the quality and credibility of the content, and on the information presented in an accessible and practical way. It is critical that SSI prevention practices reflect the best available evidence and that the evidence is as current as possible in the face of uncertainty and existing gaps in the current evidence base. Further, it is essential to include healthcare consumers such as patient representatives to ensure patients’ needs and preferences are considered during guideline development. Finally, developers need to consider the inherent challenges associated with implementation and sustainability as these have important implications for uptake and sustainability.
S1 Table. MEDLINE (EBSCO) search strategy and guideline repository searches.
S3 Table. Summary of sources where CPGs were obtained.
S4 Table. Evidence level systems used across CPGs.
- 1. World Health Organization (WHO). Implementation of the Surgical Safety Checklist. Geneva: World Health Organisation; 2008. p. 1–28.
- 2. Kassavin DS, Pascarella L, Goldfab MA. Surgical site infections: incidence and trends at a community teaching hospital. Am J Surg. 2011;201(6):749–53. pmid:21459358
- 3. European Centre for Disease Prevention and Control (ECDC). Surveillance of surgical site infections in Europe 2010–2011. Stockholm, Sweden; 2013. Contract No.: 17 July 2017.
- 4. Harbarth S, Sax H, Gastmeier P. The preventable proportion of nosocomial infections: an overview of published reports. J Hosp Infect. 2003;54(4):258–66. pmid:12919755
- 5. Mangram AJ, Horan TC, Pearson ML, Silver LC, Jarvis WR. Guideline for Prevention of Surgical Site Infection, 1999. Centers for Disease Control and Prevention (CDC) Hospital Infection Control Practices Advisory Committee. Am J Infect Control. 1999;27(2):97–132. pmid:10196487
- 6. Gillespie BM, Chaboyer W, Niewenhoven P, Rickard CM. Drivers and Barriers of Surgical Wound Management in a Large Healthcare Organisation: Results of an Environmental Scan. Wound Practices and Research: Australian Wound Management Association. 2012;20(2):90–102.
- 7. Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med 2009;6(7):e1000097. pmid:19621072
- 8. Higgins JT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0: The Cochrane Collaboration, 2011; 2011.
- 9. Smith MA, Dahlen NR. Clinical practice guideline surgical site infection prevention. Orthop Nurs. 2013;32(5):242–8. pmid:24022415
- 10. Ubbink DT, Brolmann FE, Go PM, Vermeulen H. Evidence-Based Care of Acute Wounds: A Perspective. Adv Wound Care. 2015;4(5):286–94.
- 11. Sutherland-Fraser S, Osborn S, Bryant K. Chapter One—Perioperative Nursing. In: Hamlin MD L., Richardson-Tench M. & Sutherland-Fraser S., editor. Perioperative Nursing: Elsevier; 2016.
- 12. Whittemore R, Knafl K. The integrative review: updated methodology. J Adv Nurs. 2005;52(5):546–53. pmid:16268861
- 13. Brouwers M, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, et al. AGREE II: Advancing guideline development, reporting and evaluation in healthcare. CMAJ. 2010;182(18):e839–e42. pmid:20603348
- 14. Brouwers M, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, et al. Development of the AGREE II, part 1: performance, usefulness and areas for improvement. CMAJ. 2010;182(10):1045–52. pmid:20513780
- 15. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33(1):159–74. pmid:843571
- 16. AGREE Next Steps Consortium. The AGREE II Instrument [Electronic version] Canada: AGREE Next Steps Consortium; 2009 [Available from: http://www.agreetrust.org.
- 17. AGREE Next Steps Consortium. My AGREE PLUS Canada: Canadian Institutes of Health Research 2010 [Available from: http://www.agreetrust.org/resource-centre/agree-plus/.
- 18. Anderson DJ, Kaye KS, Classen D, Arias KM, Podgorny K, Burstin H, et al. Strategies to prevent surgical site infections in acute care hospitals. Infect Control Hosp Epidemiol. 2008;29(Suppl 1):S51–61.
- 19. Bulletin of the American College of Surgeons. Guideline for prevention of surgical site infection. Bull Am Coll Surg. 2000;85(7):23–9. pmid:11349547
- 20. National Collaborating Centre for Women's and Children's Health. Surgical site infection prevention and treatment of surgical site infection. London: RCOG Press; 2008 October 2008.
- 21. World Health Organization (WHO). Global Guidelines for the Prevention of Surgical Site Infection. Geneva: World Health Organization; 2016.
- 22. Bonnar P, Dhar P, Rotstein O, Morris A, Downing M, Pearsall E, et al. Surgical Site Infection Prevention: A Clinical Practice Guidelines developed by the University of Toronto’s Best Practice in Surgery in collaboration with the Antimicrobial Stewardship Program. Canada: University of Toronto; 2017.
- 23. Berrios-Torres SI, Umscheid CA, Bratzler DW, Leas B, Stone EC, Kelz RR, et al. Centers for Disease Control and Prevention Guideline for the Prevention of Surgical Site Infection, 2017. JAMA Surg. 2017;152(8):784–91. pmid:28467526
- 24. National Institute for Health and Care Excellence. Surgical site infection—Evidence Update June 2013. Manchester, UK: National Institute for Health and Care Excellence; 2013 June 2013. Contract No.: Evidence Update 43.
- 25. Ban KA, Minei JP, Laronga C, Harbrecht BG, Jensen EH, Fry DE, et al. Executive summary of the American College of Surgeons/Surgical Infection Society Surgical Site Infection Guidelines -2016 Update. J Am Coll Surg. 2017;18(4):379–82.
- 26. Yokoe DS, Anderson DJ, Berenholtz SM, Calfee DP, Dubberke ER, Ellingson K, et al. Introduction to "A compendium of strategies to prevent healthcare-associated infections in acute care hospitals: 2014 updates". Infect Control Hosp Epidemiol. 2014;35 (Suppl 2):S1–5.
- 27. Yokoe DS, Anderson DJ, Berenholtz SM, Calfee DP, Dubberke ER, Ellingson KD, et al. A compendium of strategies to prevent healthcare-associated infections in acute care hospitals: 2014 updates. Infect Control Hosp Epidemiol. 2014;35(Suppl 2):967–77.
- 28. Yokoe DS, Classen D. Improving patient safety through infection control: a new healthcare imperative. Infect Control Hosp Epidemiol. 2008;29(Suppl 1):S3–11.
- 29. Yokoe DS, Mermel LA, Anderson DJ, Arias KM, Burstin H, Calfee DP, et al. A compendium of strategies to prevent healthcare-associated infections in acute care hospitals. Infect Control Hosp Epidemiol. 2008;29 (Suppl 1):S12–21.
- 30. Anderson DJ, Podgorny K, Berrios-Torres SI, Bratzler DW, Dellinger EP, Greene L, et al. Strategies to prevent surgical site infections in acute care hospitals: 2014 update. Infect Control Hosp Epidemiol. 2014;35(Suppl 2):S66–88.
- 31. Lo Vecchio A, Giannattasio A, Duggan C, De Masi S, Ortisi MT, Parola L, et al. Evaluation of the Quality of Guidelines for Acute Gastroenteritis in Children With the AGREE Instrument. J Pediatr Gastroenterol Nutr. 2011;52(2):183–89. pmid:20808248
- 32. Grimshaw J, Thomas RE, Maclennan G, Fraser C, Ramsay CR, Vale L, et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess 2004;8(6):1–72.
- 33. Brownlee S, Chalkidou K, Doust J, Elshaug AG, Glasziou P, Heath I, et al. Evidence for overuse of medical services around the world. Lancet. 2017;390(10090):156–68. pmid:28077234
- 34. Enia F. Clinical practice guidelines and scientific evidence. JAMA. 2009;302(2):142–7.
- 35. Gillespie BM, Chaboyer W, Kang E, Hewitt J, Niuewenhoven P, Morley N. Post-surgery wound assessment and management practices: A chart audit. J Clin Nurs. 2014;23(21–22):3250–61. pmid:24606529
- 36. Gillespie BM, Chaboyer W, St John W, Niuewenhoven P, Morley N. Health professionals’ decision-making in wound management: A grounded theory. J Adv Nurs. 2014;71(6):1238–48. pmid:25522802
- 37. Tricoci P, Allen JM, Kramer JM, Califf RM, Smith SCJ. Scientific evidence underlying the ACC/AHA clinical practice guidelines. JAMA. 2009;301(8):831–41. pmid:19244190
- 38. Casarett D. The science of choosing wisely—overcoming the therapeutic illusion. N Engl J Med. 2016;374:1203–5. pmid:27028909
- 39. Smith CA, Toupin-April K, Jutai JW, Duffy CM, Rahman P, Cavallo S, et al. A systematic critical appraisal of clinical practice guidelines in juvenile idiopathic arthritis using the appraisal of guidelines for research and evaluation II (AGREE II) instrument. PLoS One. 2015;10(9):e0137180. pmid:26356098
- 40. Li ZY, Luo L, Hu YH, Chen H, Den YK, Tang L, et al. Lung cancer screening: a systematic review of clinical practice guidelines. Int J Clin Pract. 2016;70(1):20–30. pmid:26538377
- 41. Gillespie BM, Kang E, Roberts S, Lin F, Morely N, Finigan T, et al. Reducing the risk of surgical site infection using a multidisciplinary approach: an integrative review. J Multidiscip Healthc. 2015;13(8):473–87.
- 42. Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients' care. Lancet. 2003;362(9391):1225–30. pmid:14568747
- 43. Greenhalgh T, Howick J, Maskrey N. Evidence based medicine: a movement in crisis? BMJ. 2014;348:g3725. pmid:24927763
- 44. Ocloo J, Matthews R. From tokenism to empowerment: progressing patient and public involvement in healthcare improvement. BMJ Qual Saf. 2016;25(8):626–32. pmid:26993640
- 45. Squire S, Greco M, O'Hagen B, Dickinson K, Wall D. Being patient‐centred: creating health care for our grandchildren. Clin Gov. 2006;11(1):8–46.
- 46. Gagliardi AR, Brouwers MC. Do guidelines offer implementation advice to target users? A systematic review of guideline applicability. BMJ Open. 2015;5(2).
- 47. AGREE. AGREE-REX: Recommendation EXcellence Canada: AGREE-REX Research Team; [Available from: https://www.agreetrust.org/agree-research-projects/agree-rex-recommendation-excellence/.
- 48. Gillespie BM, Chaboyer W, Erichsen-Andersson A, Hettiarachchi RM, Kularatna S. Economic case for intraoperative interventions to prevent surgical-site infection. Br J Surg. 2017;104(2):e55–e64. pmid:28121042
- 49. Centre for Reviews and Dissemination (CRD). Systematic Reviews: CRD’s guidance for undertaking reviews in health care. York, UK: CRD, University of York; 2009.
- 50. Brownlee S, Chalkidou K, Doust J, al e. Evidence for overuse of medical services around the world. The Lancet. 2017.
- 51. Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, et al. Consolidated Health Economic Evaluation Reporting Standards (CHEERS)—explanation and elaboration: a report of the ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force. Value Health. 2013;16(2):231–50. pmid:23538175