Methodological Quality of Consensus Guidelines in Implant Dentistry

Clovis Mariano Faggion Jr; Karol Apaza; Tania Ariza-Fritas; Lilian Málaga; Nikolaos Nikitas Giannakopoulos; Marco Antonio Alarcón

doi:10.1371/journal.pone.0170262

Abstract

Background

Consensus guidelines are useful to improve clinical decision making. Therefore, the methodological evaluation of these guidelines is of paramount importance. Low quality information may guide to inadequate or harmful clinical decisions.

Objective

To evaluate the methodological quality of consensus guidelines published in implant dentistry using a validated methodological instrument.

Methods

The six implant dentistry journals with impact factors were scrutinised for consensus guidelines related to implant dentistry. Two assessors independently selected consensus guidelines, and four assessors independently evaluated their methodological quality using the Appraisal of Guidelines for Research & Evaluation (AGREE) II instrument. Disagreements in the selection and evaluation of guidelines were resolved by consensus. First, the consensus guidelines were analysed alone. Then, systematic reviews conducted to support the guidelines were included in the analysis. Non-parametric statistics for dependent variables (Wilcoxon signed rank test) was used to compare both groups.

Results

Of 258 initially retrieved articles, 27 consensus guidelines were selected. Median scores in four domains (applicability, rigour of development, stakeholder involvement, and editorial independence), expressed as percentages of maximum possible domain scores, were below 50% (median, 26%, 30.70%, 41.70%, and 41.70%, respectively). The consensus guidelines and consensus guidelines + systematic reviews data sets could be compared for 19 guidelines, and the results showed significant improvements in all domain scores (p < 0.05).

Conclusions

Methodological improvement of consensus guidelines published in major implant dentistry journals is needed. The findings of the present study may help researchers to better develop consensus guidelines in implant dentistry, which will improve the quality and trust of information needed to make proper clinical decisions.

Citation: Faggion CM Jr, Apaza K, Ariza-Fritas T, Málaga L, Giannakopoulos NN, Alarcón MA (2017) Methodological Quality of Consensus Guidelines in Implant Dentistry. PLoS ONE 12(1): e0170262. https://doi.org/10.1371/journal.pone.0170262

Editor: Sompop Bencharit, Virginia Commonwealth University, UNITED STATES

Received: September 28, 2016; Accepted: December 30, 2016; Published: January 20, 2017

Copyright: © 2017 Faggion et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within the paper and its Supporting Information files.

Funding: The authors received no specific funding for this work.

Competing interests: The authors have declared that no competing interests exist.

Introduction

Consensus guidelines are important tools that help clinicians make appropriate decisions in the treatment of their patients. The developers of these guidelines suggest recommendations for clinical practice based on the best available evidence from, for example, well-conducted systematic reviews [1]. Consensus guidelines aim to promote better clinical treatment based on the weighing of potential benefits and harms, resources available, patients’ preferences, and scientific evidence [2]. To reach this goal, guidelines should be developed at the highest methodological level possible. Those based on low-quality, biased methodologies will likely guide clinicians to make ineffective and potentially harmful clinical decisions. Thus, evaluation of the methodological quality of consensus guidelines is important in any field.

Consensus guidelines are usually planned and developed by leading experts in the respective field who personally meet to discuss recommendations for clinical practice. As the name suggests, there is always an attempt for reaching consensus in generating clinical recommendations. This approach does not vary too much from the development of so-called “classic guidelines” which are also produced by specialists in the field of the respective guideline. For example, the Cochrane Collaboration defines a clinical guideline as “a systematically developed statement for practitioners and participants about appropriate health care for specific clinical circumstances” [http://community-archive.cochrane.org/glossary/]. Thus, although one can argue that might have slight methodological differences between a consensus and a “classic” guideline, the objective of both guidelines is exactly the same: the improvement of health care of our patients. Therefore, both documents require to be scrutinized for their quality.

The Appraisal of Guidelines Research and Evaluation (AGREE) tool is a validated instrument used to evaluate the methodological quality and transparency of development of clinical guidelines. This tool was first published in 2003 [3], and the most recent version (AGREE II) [4] has been refined and updated with better methodological properties [5]. Both versions of the instrument have been used in a variety of medical disciplines [6–8]. To our knowledge, however, the methodological quality of consensus guidelines in implant dentistry has not been evaluated.

The main objective of this study was to evaluate the methodological quality of consensus guidelines published in highly ranked implant dentistry journals using the AGREE II tool. A secondary objective was to evaluate whether the inclusion of systematic reviews conducted to support the consensus guidelines improved the methodological quality of the consensus guidelines.

Material and Methods

Study question

This methodological study was performed to answer the question, “Do consensus guidelines published in highly ranked implant dentistry journals meet the requirements proposed in the AGREE II instrument?”

Eligibility criteria

Consensus guidelines on implant dentistry published since 2009 (with the respective consensus conference held after May 2009) in the six major implant dentistry journals (listed below) were included. Other types of document, such as those related to primary and secondary research, were excluded. Data from systematic reviews conducted to support the consensus guidelines were also included in the second part of the assessment.

Search

Two authors (KA, MA) independently searched for consensus guidelines in the six implant dentistry journals with impact factors (2014) assigned by Journal Citation Reports (http://ipscience.thomsonreuters.com/product/journal-citation-reports/?utm_source=false&utm_medium=false&utm_campaign=false): Clinical Oral Implants Research (COIR), Clinical Implant Dentistry and Related Research (CIDRR), European Journal of Oral Implants (EJOI), The International Journal of Oral and Maxillofacial Implants (JOMI), Journal of Oral Implantology, and Implant Dentistry. Searches were limited to guidelines published between May 2009 and February 2016. The Medline database was also searched (via PubMed), using the following key words and Boolean operators: ‘guidelines’ OR ‘consensus’ OR ‘position paper’ OR ‘workshop’ OR ‘proceeding’ OR ‘conference’ in combination (AND) with each of the six journal titles. This second search was conducted to provide a detailed pathway for reporting of the literature search process.

Selection of reports

First, two authors (KA, MA) evaluated the titles and abstracts of reports to determine eligibility for initial inclusion. Then, they scrutinised full texts of papers to determine whether the studies met the inclusion criteria. The authors documented excluded articles, with corresponding reasons for exclusion. The two authors performed study selection independently and in duplicate, and discussed any disagreement regarding the inclusion or exclusion of papers until consensus was achieved.

The AGREE II instrument

The AGREE II tool is an updated version of the seminal AGREE tool developed by the AGREE Collaboration [3], a group of researchers and guideline developers. It consists of 23 items in six domains (Table 1), used mainly to evaluate the methodological rigour and transparency of guidelines [5]. Items are rated using a seven-point scale ranging from ‘strongly disagree’ to ‘strongly agree’, representing the assessor’s confidence in whether the guidelines meet the quality of reporting and AGREE criteria. Each domain score is calculated by summing component item scores and scaling the value as a percentage of the maximum possible score, according to the developer’s instructions. As the AGREE II tool was made publicly available in May 2009, only consensus guidelines published from this year forward (with the respective consensus conference held after May 2009) were included in the present study.

Download:

Table 1. AGREE II items.

https://doi.org/10.1371/journal.pone.0170262.t001

Data evaluation

Four authors (KA, TA, LM, and MA) independently applied the AGREE II tool, first to consensus guidelines only, and then with the inclusion of systematic reviews conducted to support the guidelines. The latter assessment was performed to understand the amount of information added to clinical recommendations by the consideration of systematic reviews as supporting material. Disagreements on data evaluation were resolved by discussion among the four authors until consensus was achieved.

Assessor training

A standardised form containing the 23 AGREE II items was produced for data extraction/evaluation. After carefully reading the AGREE handbook, the four assessors applied the tool to evaluate the methodology of consensus guidelines not included in the present study, recording data in the form. Between rounds of data evaluation, assessors discussed the outcomes comprehensively to improve the homogeneity of assessment.

Data analysis

Domains scores were presented as medians of percentages of maximum possible scores with their respective interquartile range (IQR). Domain scores from the two data sets (consensus guidelines and consensus guidelines plus supporting systematic reviews) were compared using non-parametric statistics for dependent variables (Wilcoxon signed rank test), with the level of significance set at p = 0.05. Statistical analyses were performed with the SigmaPlot software (version 12.0 for Windows; Systat Software GmbH, Erkrath, Germany).

Results

Number of consensus guidelimatic reviews supporting the guidelines. t/disagreement ratings between consensus guidelines and consensus guidnes

We initially identified 258 publications. After the assessment of titles and abstracts, 213 publications were excluded. Full text evaluation led to the exclusion of 45 additional publications. Hence, 27 consensus guidelines were included. The literature search process is illustrated in Fig 1, and publications included in and excluded from the analysis are listed in the supplementary information in S1 and S2 Appendix, respectively.

Download:

Fig 1. PRISMA-style flow diagram of the consensus guidelines search process.

https://doi.org/10.1371/journal.pone.0170262.g001

Characteristics of consensus guidelines

Consensus guidelines were published in five of the six journals searched: COIR (n = 12), JOMI (n = 7), EJOI (n = 6), CIDRR (n = 1), and Implant Dentistry (n = 1). Twenty-six guidelines were developed after meetings held in European countries. The number of authors of the consensus guidelines ranged from 2 to 27 (median, 9). The European Association for Osseointegration was the organisation that most frequently supported the meetings and development of consensus guidelines (n = 9). Table 2 provides detailed information on the characteristics of consensus guidelines included in this study.

Download:

Table 2. Characteristics of the consensus guidelines included in this study.

https://doi.org/10.1371/journal.pone.0170262.t002

Methodological quality of guidelines

Consensus guidelines only.

The score for domain 4 (clarity of presentation) was highest (median, 75; IQR 15.30), followed by the score for domain 1 (scope and purpose: median, 69.40; IQR, 36.20). Median scores for domains 2 (stakeholder involvement) and 6 (editorial independence) were both 41.70 (IQRs, 17.70 and 83.30, respectively). Scores for domains 3 (rigour of development) and 5 (applicability) were lowest (median, 30.70 [IQR, 26.50] and 26 [IQR, 12.50], respectively).

Consensus guidelines plus systematic reviews.

When systematic reviews were included in the sample, the score for domain 1 was highest (median, 84.70; IQR, 9.80). The median score for domain 6 was second highest (79.20; IQR, 73), but this score showed the greatest variability among guidelines. The third highest score was for domain 4 (median, 76.40; IQR, 18.10), followed by the scores for domains 3 and 2 (median, 56.30 [IQR, 34.40] and 50 [IQR, 44.40], respectively). The score for domain 5 was lowest (median, 26; IQR, 20.80). Tables in S2 and S3 Tables report the complete AGREE II scores for consensus guidelines alone and consensus guidelines + systematic reviews, respectively. Table in S3 Table shows domain scores for the 19 consensus guidelines with no difference in hierarchy between data sets.

Group comparisons.

Domain scores could be compared between data sets for 19 of the 27 consensus guidelines (Table in S4 Table). Scores differed significantly for all domains (p < 0.05; Table 3).

Download:

Table 3. Comparison of AGREE II scores (percentage of the maximum possible score for a specific domain) between consensus guidelines alone and consensus guidelines + systematic reviews.

CG: consensus guideline; CGSR: consensus guideline + systematic review). Scores are related to 19 possible comparisons. The p values refer to Wilcoxon Signed Rank Test for paired samples.

https://doi.org/10.1371/journal.pone.0170262.t003

Discussion

Brief summary of findings

In this sample of 27 consensus guidelines in implant dentistry, median scores for four AGREE II domains (stakeholder involvement, rigour of development, applicability, and editorial independence) were less than 50. However, the inclusion of supporting systematic reviews significantly improved all domain scores. Great variability was found among consensus guidelines, as reflected by large IQRs for some domains.

Implications of the present findings

The present findings have important consequences for the further development of consensus guidelines in implant dentistry. First, they provide a measurement of the methodological quality of these guidelines that may greatly impact clinicians’ decisions. Consensus guidelines in the present sample were supported by reputable implant dentistry organisations, and were published in highly ranked implant dentistry journals, which are reliable sources of information for clinicians working with dental implants. Second, the findings provide comprehensive information about which domains should be prioritised in the development of future guidelines in this field. Third, this study demonstrated that the AGREE II tool can serve as a reference for the development of future consensus guidelines in implant dentistry.

In the present study, scores for domain 5 (applicability) were lowest. These results show that a gap currently exists between the evidence provided and its applicability in the clinical setting. Scores for domain 3 (rigour of development), which more directly reflects the methodological aspects of the guidelines, were second lowest. Importantly, scores for domain 2 (stakeholder involvement) were also low. For example, the sub-item ‘the views and preferences of the target population (patients, public, etc.) have been sought’ was poorly addressed in all consensus guidelines. Patients’ views are pivotal in gaining an understanding of their needs, and future guidelines in implant dentistry should include more information from patients’ perspectives. One approach would be to select patients to attend or participate in consensus meetings.

The significant improvement in all domain scores achieved by the inclusion of systematic reviews suggests that these reviews contain much important information needed to evaluate the methodological quality of consensus guidelines. We thus recommend that users examine both types of material to more fully understand the quality of guidelines. Ideally, systematic reviews and guidelines are produced at the highest methodological level possible and published separately, with the reviews serving as the source for guideline development [1].

Comparison with other studies

Few publications describe the use of the AGREE II tool to evaluate clinical guidelines in dentistry. Horner et al. [9] recently evaluated 26 guidelines on the use of cone-beam computerised tomography in dental and maxillofacial radiology using the AGREE II instrument. As in the present analysis, they obtained good scores for domain 1 (scope and purpose) and very poor scores for domain 5 (applicability) [9]. San Martin-Galindo et al. [10] used the AGREE II tool to evaluate three guidelines on the use of pit and fissure sealants for dental clinicians; scores for domain 6 (editorial independence) were lowest. In the present study, this domain score was the third lowest when the consensus guidelines were evaluated alone. These findings may reflect the lack of good reporting of potential conflicts of interest by parties involved in guideline development. A few other studies have evaluated guidelines in dentistry using the original AGREE instrument [6,11–14]. Most of these studies showed that guidelines were of low quality.

Strengths and limitations of the present study

To our knowledge, this study is the first to evaluate the methodological quality of consensus guidelines published in highly ranked implant dentistry journals using a validated tool. The AGREE II instrument represents improvement over the original AGREE tool, enabling more in-depth evaluation of the strengths and weaknesses of guidelines, and it has shown validity and reliability [15,16]. Thus, this evaluation of the quality of consensus guidelines was probably conducted with the best methodological tool available.

One may argue that AGREE II instrument is an inadequate methodology for assessing consensus guidelines. The idea is that consensus guidelines are developed by experts attending workshops and they do not fulfil the requirements of a quality instrument. But, this is the main reason for applying an instrument such as AGREE II. The question here is: what is the validity of a document that does not allow audit and quality evaluation? Therefore, we understand this approach is appropriate for several reasons. Firstly, guidelines included in the study fall within the Cochrane Collaboration’s definition of a clinical guideline as ‘a systematically developed statement for practitioners and participants about appropriate health care for specific clinical circumstances’ [17]. Secondly, the AGREE II handbook reports that the instrument is “generic” and can be applied to a great variety of documents. Thirdly, the literature contains several reports on the use of the AGREE instrument to evaluate consensus guidelines in other medical fields [8, 18–24]. Fourthly, and finally, the consensus guidelines included in this sample were developed by key-people in the respective field, and dental practitioners will likely follow them to make clinical decisions. So, the effect is the same of a considered “standard” guideline. In the end, clinicians will use the document for improving clinical treatments. Hence, the focus should be on whether the document includes recommendations for clinical action, instead on its structure or how it was developed.

We statistically compared evaluations performed with and without additional information from systematic reviews supporting the consensus guidelines. However, some limitations should be kept in mind when interpreting these results. Firstly, adequate sample size was difficult to determine, and our sample of guidelines is arguably small. Nevertheless, AGREE II domain scores showed robust and significant improvement with the inclusion of data from systematic reviews, and similar results would likely be obtained with a larger sample. Secondly, comparison is ideally performed between two independent groups. However, the identification of two sets of similar guidelines (in terms of structure and objectives) for a comparison like that performed in this study would be challenging. Although limited, this comparison is relevant because it provides quantitative evidence for the amount of information that supporting systematic reviews can add to consensus guidelines. In other words, the reader can understand these two different scenarios (guidelines with and without systematic review). More than focusing on p values, which might generate misleading assumptions [25], readers should observe the magnitude of changes in AGREE II scores.

In the present study, we did not attempt to determine which consensus guidelines are recommended for clinical practice and which are not. As reported in the AGREE II user’s manual (AGREE II instrument), the AGREE Collaboration does not recommend the application of any score threshold to differentiate between high-quality and poor-quality guidelines. They recommend that decisions about the use of guidelines be made by users, oriented by the context in which the AGREE II instrument is applied.

Future development of consensus guidelines in implant dentistry

The consensus guidelines included in the present study were produced by key opinion leaders in the field of implant dentistry. We understand that the involvement of authorities, researchers, and clinicians in the development of such guidelines is important, as it represents integration between research foundation and clinical relevance. However, guidelines should be produced to the highest methodological quality possible, to give users more accurate information about the level and quality of evidence they intend to apply in the clinical setting.

Implant dentistry has reached a level of excellence in the conducting of systematic reviews. Now, it is time to move forward to improve the quality of clinical guidelines, which can provide a bridge between evidence and its applicability. The concept of developing consensus guidelines without a robust methodology is a remnant from the “pre–evidence-based” era. Hence, this methodological gap between well-developed systematic reviews and clinical practice guidelines should be reduced.

The so-called “classic” guidelines should also be scrutinized for quality with the AGREE II instrument. In the implant dentistry field they may also be available. For example, we searched Medline (via PubMed) for such guidelines (search strategy: dent*[ti] AND implant*[ti] AND guideline*[ti] AND 2009: 2016[dp], in 25th December 2016), and found 14 potential “non-consensus” implant dentistry guidelines. Although it is not in the scope of the present study to evaluate these guidelines, it would be also important to evaluate them in a future project.

Conclusions

There is room to improve the quality of consensus guidelines published in highly ranked implant dentistry journals. Clinicians’ and researchers’ development of consensus guidelines to improve clinical treatment with dental implants is laudable. However, as for primary and secondary research, these guidelines should adhere to high and transparent standards. The AGREE II instrument can be used as a reference for the development of high-quality guidelines to provide unbiased and adequate clinical recommendations to clinicians working with dental implants.

Supporting Information

S1 PRISMA Checklist.

https://doi.org/10.1371/journal.pone.0170262.s001

(DOC)

S1 Appendix. List of included consensus guidelines.

https://doi.org/10.1371/journal.pone.0170262.s002

(DOCX)

S2 Appendix. List of excluded documents with reasons for exclusion.

https://doi.org/10.1371/journal.pone.0170262.s003

(DOCX)

S1 Table. Evaluation of consensus guidelines published in high-ranked implant dentistry journals with the AGREE II instrument.

https://doi.org/10.1371/journal.pone.0170262.s004

(DOCX)

S2 Table. Evaluation of consensus guidelines + systematic review published in high-ranked implant dentistry journals with the AGREE II instrument.

https://doi.org/10.1371/journal.pone.0170262.s005

(DOCX)

S3 Table. Medians of percentages of the maximum possible score for the respective domains across consensus guidelines in implant dentistry (19 possible comparisons).

IQR: interquartile range. CG: consensus guideline; CGSR: consensus guideline + systematic review.

https://doi.org/10.1371/journal.pone.0170262.s006

(DOCX)

S4 Table. Percentage of the maximum possible score for the respective domains across consensus guidelines in implant dentistry

https://doi.org/10.1371/journal.pone.0170262.s007

(DOCX)

Author Contributions

Conceptualization: CMF MA.
Data curation: CMF KA TAF LM NNG MA.
Formal analysis: CMF NNG MA.
Funding acquisition: CMF.
Investigation: CMF KA TAF LM NNG MA.
Methodology: CMF MA.
Project administration: CMF.
Resources: CMF NNG MA.
Software: CMF NNG.
Supervision: CMF.
Validation: CMF KA TAF LM NNG MA.
Visualization: CMF KA TAF LM NNG MA.
Writing – original draft: CMF.
Writing – review & editing: CMF KA TAF LM NNG MA.

References

1. Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011;64: 383–394. pmid:21195583
- View Article
- PubMed/NCBI
- Google Scholar
2. Guyatt GH, Oxman AD, Kunz R, Falck-Ytter Y, Vist GE, Liberati A, et al. Going from evidence to recommendations. BMJ. 2008;336: 1049–1051. pmid:18467413
- View Article
- PubMed/NCBI
- Google Scholar
3. AGREE Collaboration. Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care. 2003;12: 18–23. pmid:12571340
- View Article
- PubMed/NCBI
- Google Scholar
4. AGREE II instrument. Available: http://www.agreetrust.org/wp-content/uploads/2013/10/AGREE-II-Users-Manual-and-23-item-Instrument_2009_UPDATE_2013.pdf. Accessed on 27 June 2016.
5. Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, et al. AGREE II: advancing guideline development, reporting and evaluation in health care. CMAJ. 2010;182: E839–842. pmid:20603348
- View Article
- PubMed/NCBI
- Google Scholar
6. Faggion CM Jr. Clinician assessment of guidelines that support common dental procedures. J Evid Based Dent Pract. 2008;8: 1–7. pmid:18346691
- View Article
- PubMed/NCBI
- Google Scholar
7. Barriocanal AM, López A, Monreal M, Montané E. Quality assessment of peripheral artery disease clinical guidelines. J Vasc Surg. 2016;63:1091–1098. pmid:27016858
- View Article
- PubMed/NCBI
- Google Scholar
8. Deng W, Li L, Wang Z, Chang X, Li R, Fang Z, et al. Using AGREE II to Evaluate the Quality of Traditional Medicine Clinical Practice Guidelines in China. J Evid Based Med. 2016 Mar 15.
- View Article
- Google Scholar
9. Horner K, O'Malley L, Taylor K, Glenny AM. Guidelines for clinical use of CBCT: a review. Dentomaxillofac Radiol. 2015;44: 20140225. pmid:25270063
- View Article
- PubMed/NCBI
- Google Scholar
10. San Martin-Galindo L, Rodríguez-Lozano FJ, Abalos-Labruzzi C, Niederman R.European Fissure Sealant Guidelines: assessment using AGREE II. Int J Dent Hyg. 2015 Sep 11.
- View Article
- Google Scholar
11. Glenny AM, Worthington HV, Clarkson JE, Esposito M. The appraisal of clinical guidelines in dentistry. Eur J Oral Implantol. 2009;2: 135–143. pmid:20467612
- View Article
- PubMed/NCBI
- Google Scholar
12. Parnell C, Whelton H, O'Mullane D. Water fluoridation. Eur Arch Paediatr Dent. 2009;10: 141–148. pmid:19772843
- View Article
- PubMed/NCBI
- Google Scholar
13. van Diermen DE, Aartman IH, Baart JA, Hoogstraten J, van der Waal I. Dental management of patients using antithrombotic drugs: critical appraisal of existing guidelines. Oral Surg Oral Med Oral Pathol Oral Radiol Endod. 2009;107: 616–624. pmid:19426918
- View Article
- PubMed/NCBI
- Google Scholar
14. Shah P, Moles DR, Parekh S, Ashley P, Siddik D. Evaluation of pediatric dentistry guidelines using the AGREE instrument. Pediatr Dent. 2011;33: 120–129. pmid:21703061
- View Article
- PubMed/NCBI
- Google Scholar
15. Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, et al. Development of the AGREE II, part 2: assessment of validity of items and tools to support application. CMAJ. 2010;182: E472–478. pmid:20513779
- View Article
- PubMed/NCBI
- Google Scholar
16. Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, et al. Development of the AGREE II, part 1: performance, usefulness and areas for improvement. CMAJ. 2010;182: 1045–1052. pmid:20513780
- View Article
- PubMed/NCBI
- Google Scholar
17. Cochrane Glossary. Available: https://community-archive.cochrane.org/glossary. Accessed on 27 June 2016.
18. Lopez-Olivo MA, Kallen MA, Ortiz Z, Skidmore B, Suarez-Almazor ME. Quality appraisal of clinical practice guidelines and consensus statements on the use of biologic agents in rheumatoid arthritis: a systematic review. Arthritis Rheum. 2008;59: 1625–1638. pmid:18975351
- View Article
- PubMed/NCBI
- Google Scholar
19. Guo J, Cheng C, Yan W, Xu G, Feng J, Wang T, et al. Systematic review of clinical practice guidelines related to multiple sclerosis. PLoS One. 2014;9:e106762. pmid:25302678
- View Article
- PubMed/NCBI
- Google Scholar
20. Nagler EV, Vanmassenhove J, van der Veer SN, Nistor I, Van Biesen W, Webster AC, et al. Diagnosis and treatment of hyponatremia: a systematic review of clinical practice guidelines and consensus statements. BMC Med. 2014;12: 1.
- View Article
- Google Scholar
21. Jacobs C, Graham ID, Makarski J, Chassé M, Fergusson D, Hutton B, et al. Clinical practice guidelines and consensus statements in oncology—an assessment of their methodological quality. PLoS One. 2014;9: e110469. eCollection 2014. pmid:25329669
- View Article
- PubMed/NCBI
- Google Scholar
22. Zhang W, Moskowitz RW, Nuki G, Abramson S, Altman RD, Arden N, et al. OARSI recommendations for the management of hip and knee osteoarthritis, Part II: OARSI evidence-based, expert consensus guidelines. Osteoarthritis Cartilage. 2008;16: 137–162. pmid:18279766
- View Article
- PubMed/NCBI
- Google Scholar
23. Wang Y, Luo Q, Li Y, Wang H, Deng S, Wei S, et al. Quality assessment of clinical practice guidelines on the treatment of hepatocellular carcinoma or metastatic liver cancer. PLoS One. 2014;9: e103939. pmid:25105961
- View Article
- PubMed/NCBI
- Google Scholar
24. Shawyer AC, Livingston MH, Manja V, Brouwers MC. The quality of guidelines in pediatric surgery: can we all AGREE? Pediatr Surg Int. 2015;31: 61–68. pmid:25336247
- View Article
- PubMed/NCBI
- Google Scholar
25. Nuzzo R. Scientific method: statistical errors. Nature. 2014;506:150–152. pmid:24522584
- View Article
- PubMed/NCBI
- Google Scholar

[ref1] 1. Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011;64: 383–394. pmid:21195583
View Article
PubMed/NCBI
Google Scholar

[2] View Article

[3] PubMed/NCBI

[4] Google Scholar

[ref2] 2. Guyatt GH, Oxman AD, Kunz R, Falck-Ytter Y, Vist GE, Liberati A, et al. Going from evidence to recommendations. BMJ. 2008;336: 1049–1051. pmid:18467413
View Article
PubMed/NCBI
Google Scholar

[6] View Article

[7] PubMed/NCBI

[8] Google Scholar

[ref3] 3. AGREE Collaboration. Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care. 2003;12: 18–23. pmid:12571340
View Article
PubMed/NCBI
Google Scholar

[10] View Article

[11] PubMed/NCBI

[12] Google Scholar

[ref4] 4. AGREE II instrument. Available: http://www.agreetrust.org/wp-content/uploads/2013/10/AGREE-II-Users-Manual-and-23-item-Instrument_2009_UPDATE_2013.pdf. Accessed on 27 June 2016.

[ref5] 5. Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, et al. AGREE II: advancing guideline development, reporting and evaluation in health care. CMAJ. 2010;182: E839–842. pmid:20603348
View Article
PubMed/NCBI
Google Scholar

[15] View Article

[16] PubMed/NCBI

[17] Google Scholar

[ref6] 6. Faggion CM Jr. Clinician assessment of guidelines that support common dental procedures. J Evid Based Dent Pract. 2008;8: 1–7. pmid:18346691
View Article
PubMed/NCBI
Google Scholar

[19] View Article

[20] PubMed/NCBI

[21] Google Scholar

[ref7] 7. Barriocanal AM, López A, Monreal M, Montané E. Quality assessment of peripheral artery disease clinical guidelines. J Vasc Surg. 2016;63:1091–1098. pmid:27016858
View Article
PubMed/NCBI
Google Scholar

[23] View Article

[24] PubMed/NCBI

[25] Google Scholar

[ref8] 8. Deng W, Li L, Wang Z, Chang X, Li R, Fang Z, et al. Using AGREE II to Evaluate the Quality of Traditional Medicine Clinical Practice Guidelines in China. J Evid Based Med. 2016 Mar 15.
View Article
Google Scholar

[27] View Article

[28] Google Scholar

[ref9] 9. Horner K, O'Malley L, Taylor K, Glenny AM. Guidelines for clinical use of CBCT: a review. Dentomaxillofac Radiol. 2015;44: 20140225. pmid:25270063
View Article
PubMed/NCBI
Google Scholar

[30] View Article

[31] PubMed/NCBI

[32] Google Scholar

[ref10] 10. San Martin-Galindo L, Rodríguez-Lozano FJ, Abalos-Labruzzi C, Niederman R.European Fissure Sealant Guidelines: assessment using AGREE II. Int J Dent Hyg. 2015 Sep 11.
View Article
Google Scholar

[34] View Article

[35] Google Scholar

[ref11] 11. Glenny AM, Worthington HV, Clarkson JE, Esposito M. The appraisal of clinical guidelines in dentistry. Eur J Oral Implantol. 2009;2: 135–143. pmid:20467612
View Article
PubMed/NCBI
Google Scholar

[37] View Article

[38] PubMed/NCBI

[39] Google Scholar

[ref12] 12. Parnell C, Whelton H, O'Mullane D. Water fluoridation. Eur Arch Paediatr Dent. 2009;10: 141–148. pmid:19772843
View Article
PubMed/NCBI
Google Scholar

[41] View Article

[42] PubMed/NCBI

[43] Google Scholar

[ref13] 13. van Diermen DE, Aartman IH, Baart JA, Hoogstraten J, van der Waal I. Dental management of patients using antithrombotic drugs: critical appraisal of existing guidelines. Oral Surg Oral Med Oral Pathol Oral Radiol Endod. 2009;107: 616–624. pmid:19426918
View Article
PubMed/NCBI
Google Scholar

[45] View Article

[46] PubMed/NCBI

[47] Google Scholar

[ref14] 14. Shah P, Moles DR, Parekh S, Ashley P, Siddik D. Evaluation of pediatric dentistry guidelines using the AGREE instrument. Pediatr Dent. 2011;33: 120–129. pmid:21703061
View Article
PubMed/NCBI
Google Scholar

[49] View Article

[50] PubMed/NCBI

[51] Google Scholar

[ref15] 15. Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, et al. Development of the AGREE II, part 2: assessment of validity of items and tools to support application. CMAJ. 2010;182: E472–478. pmid:20513779
View Article
PubMed/NCBI
Google Scholar

[53] View Article

[54] PubMed/NCBI

[55] Google Scholar

[ref16] 16. Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, et al. Development of the AGREE II, part 1: performance, usefulness and areas for improvement. CMAJ. 2010;182: 1045–1052. pmid:20513780
View Article
PubMed/NCBI
Google Scholar

[57] View Article

[58] PubMed/NCBI

[59] Google Scholar

[ref17] 17. Cochrane Glossary. Available: https://community-archive.cochrane.org/glossary. Accessed on 27 June 2016.

[ref18] 18. Lopez-Olivo MA, Kallen MA, Ortiz Z, Skidmore B, Suarez-Almazor ME. Quality appraisal of clinical practice guidelines and consensus statements on the use of biologic agents in rheumatoid arthritis: a systematic review. Arthritis Rheum. 2008;59: 1625–1638. pmid:18975351
View Article
PubMed/NCBI
Google Scholar

[62] View Article

[63] PubMed/NCBI

[64] Google Scholar

[ref19] 19. Guo J, Cheng C, Yan W, Xu G, Feng J, Wang T, et al. Systematic review of clinical practice guidelines related to multiple sclerosis. PLoS One. 2014;9:e106762. pmid:25302678
View Article
PubMed/NCBI
Google Scholar

[66] View Article

[67] PubMed/NCBI

[68] Google Scholar

[ref20] 20. Nagler EV, Vanmassenhove J, van der Veer SN, Nistor I, Van Biesen W, Webster AC, et al. Diagnosis and treatment of hyponatremia: a systematic review of clinical practice guidelines and consensus statements. BMC Med. 2014;12: 1.
View Article
Google Scholar

[70] View Article

[71] Google Scholar

[ref21] 21. Jacobs C, Graham ID, Makarski J, Chassé M, Fergusson D, Hutton B, et al. Clinical practice guidelines and consensus statements in oncology—an assessment of their methodological quality. PLoS One. 2014;9: e110469. eCollection 2014. pmid:25329669
View Article
PubMed/NCBI
Google Scholar

[73] View Article

[74] PubMed/NCBI

[75] Google Scholar

[ref22] 22. Zhang W, Moskowitz RW, Nuki G, Abramson S, Altman RD, Arden N, et al. OARSI recommendations for the management of hip and knee osteoarthritis, Part II: OARSI evidence-based, expert consensus guidelines. Osteoarthritis Cartilage. 2008;16: 137–162. pmid:18279766
View Article
PubMed/NCBI
Google Scholar

[77] View Article

[78] PubMed/NCBI

[79] Google Scholar

[ref23] 23. Wang Y, Luo Q, Li Y, Wang H, Deng S, Wei S, et al. Quality assessment of clinical practice guidelines on the treatment of hepatocellular carcinoma or metastatic liver cancer. PLoS One. 2014;9: e103939. pmid:25105961
View Article
PubMed/NCBI
Google Scholar

[81] View Article

[82] PubMed/NCBI

[83] Google Scholar

[ref24] 24. Shawyer AC, Livingston MH, Manja V, Brouwers MC. The quality of guidelines in pediatric surgery: can we all AGREE? Pediatr Surg Int. 2015;31: 61–68. pmid:25336247
View Article
PubMed/NCBI
Google Scholar

[85] View Article

[86] PubMed/NCBI

[87] Google Scholar

[ref25] 25. Nuzzo R. Scientific method: statistical errors. Nature. 2014;506:150–152. pmid:24522584
View Article
PubMed/NCBI
Google Scholar

[89] View Article

[90] PubMed/NCBI

[91] Google Scholar

Correction

Figures

Abstract

Background

Objective

Methods

Results

Conclusions

Introduction

Material and Methods

Study question

Eligibility criteria

Search

Selection of reports

The AGREE II instrument

Data evaluation

Assessor training

Data analysis

Results

Number of consensus guidelimatic reviews supporting the guidelines. t/disagreement ratings between consensus guidelines and consensus guidnes

Characteristics of consensus guidelines

Methodological quality of guidelines

Consensus guidelines only.

Consensus guidelines plus systematic reviews.

Group comparisons.

Discussion

Brief summary of findings

Implications of the present findings

Comparison with other studies

Strengths and limitations of the present study

Future development of consensus guidelines in implant dentistry

Conclusions

Supporting Information

S1 PRISMA Checklist.

S1 Appendix. List of included consensus guidelines.

S2 Appendix. List of excluded documents with reasons for exclusion.

S1 Table. Evaluation of consensus guidelines published in high-ranked implant dentistry journals with the AGREE II instrument.

S2 Table. Evaluation of consensus guidelines + systematic review published in high-ranked implant dentistry journals with the AGREE II instrument.

S3 Table. Medians of percentages of the maximum possible score for the respective domains across consensus guidelines in implant dentistry (19 possible comparisons).

S4 Table. Percentage of the maximum possible score for the respective domains across consensus guidelines in implant dentistry

Author Contributions

References