The Quality of the Evidence According to GRADE Is Predominantly Low or Very Low in Oral Health Systematic Reviews

Nikolaos Pandis; Padhraig S. Fleming; Helen Worthington; Georgia Salanti

doi:10.1371/journal.pone.0131644

Abstract

Objectives

The main objective was to assess the credibility of the evidence using Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) in oral health systematic reviews on the Cochrane Database of Systematic Reviews (CDSR) and elsewhere.

Study Design and Setting

Systematic Reviews or meta-analyses (January 2008-December 2013) from 14 high impact general dental and specialty dental journals and the Cochrane Database of Systematic Reviews were screened for meta-analyses. Data was collected at the systematic review, meta-analysis and trial level. Two reviewers applied and agreed on the GRADE rating for the selected meta-analyses.

Results

From the 510 systematic reviews initially identified 91 reviews (41 Cochrane and 50 non-Cochrane) were eligible for inclusion. The quality of evidence was high in 2% and moderate in 18% of the included meta-analyses with no difference between Cochrane and non-Cochrane reviews, journal impact factor or year of publication. The most common domains prompting downgrading of the evidence were study limitations (risk of bias) and imprecision (risk of play of chance).

Conclusion

The quality of the evidence in oral health assessed using GRADE is predominantly low or very low suggesting a pressing need for more randomised clinical trials and other studies of higher quality in order to inform clinical decisions thereby reducing the risk of instituting potentially ineffective and/or harmful therapies.

Citation: Pandis N, Fleming PS, Worthington H, Salanti G (2015) The Quality of the Evidence According to GRADE Is Predominantly Low or Very Low in Oral Health Systematic Reviews. PLoS ONE 10(7): e0131644. https://doi.org/10.1371/journal.pone.0131644

Editor: Christian Gluud, Copenhagen University Hospital, DENMARK

Received: February 16, 2015; Accepted: June 5, 2015; Published: July 10, 2015

Copyright: © 2015 Pandis et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited

Data Availability: All relevant data are within the paper and its Supporting Information files.

Funding: This study was supported by a grant of the Papavramidès Foundation, Bern, Switzerland to the first author. Additionally, Georgia Salanti received funding from the European Research Council (ERC Starting Grant IMMA 260559). These funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Introduction

Systematic reviews aim to assimilate high-quality evidence on the area of interest in a systematic, transparent, and unbiased manner leading to qualitative or quantitative synthesis. Quantitative synthesis can produce a more precise estimate on the efficacy and safety of a therapy, and can reconcile misunderstandings and controversies, and form the basis for future trials [1]. Healthcare practitioners should seek high quality evidence when searching for answers to clinical questions in relation to the effectiveness or otherwise of a proposed intervention. Several meta-epidemiological studies have been published on the quality of systematic reviews in oral health, with evidence to suggest both reporting and methodological deficiencies common to systematic reviews within a range of specialty areas [2–5]; these findings have also mirrored analogous research studies in other biomedical areas [6,7]. However, the overall quality of the existing evidence in oral health has not yet been assessed. Moreover, there is increasing concern of a gulf between research evidence and its clinical applicability. Patients are not typically conversant in the scientific publications but are exercised by the implications of research findings on their lives and wellbeing. It has become evident that a system capable of simultaneously assessing the quality of the evidence, balancing benefits and harms, while accounting for patient preferences and aiding clear treatment recommendations is imperative [8]. This approach would resonate both within medicine and in dentistry where an increasing number and quality of systematic reviews are published.

Several groups have proposed complex methods for evaluating and translating evidence into clinical practice; many of these have been somewhat confusing and impractical [9]. The GRADE (Grades of Recommendation, Assessment, Development, and Evaluation) initiative, however, has become an accepted approach for assessing the evidence and consequently making recommendations [8]. The GRADE approach is predicated on a precise clinical question, with consideration of all important outcomes within a systematic review prioritizing them based on their relative importance [10]. Subsequently, the existing quality of the evidence for an intervention and a particular outcome from a systematic review is assessed based on a specific protocol and graded as high, moderate, low or very low (Table 1) [11].

Download:

Table 1. Categories of quality of evidence according to GRADE.

A single downgrade (for randomized clinical trials) or upgrade (for non-randomized studies) corresponds to one level change in the ranking.

https://doi.org/10.1371/journal.pone.0131644.t001

While, precursors relied heavily on the overall study design (randomized compared to non-randomized studies) to evaluate a body of evidence, study design remains important within the GRADE assessment, although not the sole arbiter of the quality of evidence. Randomized clinical trials provide a higher quality of evidence compared to observational studies; however, good quality observational studies are more valuable than uncontrolled case series designs [12]. Randomized clinical trials (RCTs) are initially rated as high quality but may be downgraded after accounting for study limitations, indirectness, inconsistency, imprecision, and publication bias. Observational studies start at low quality but may be upgraded in the presence of a large magnitude of effect after potential confounding and dose-response effects have been considered. In the GRADE system, although expert opinion does not command a quality of evidence score, it is considered critical in interpreting, combining and placing the available evidence in the correct context [12].

The GRADE approach has been endorsed by a large number of societies and institutions including The Cochrane Collaboration [13] and has been used in various fields of medicine such as allergies, heart disease, oncology, endocrinology and respiratory and critical care medicine [14–17]. GRADE has also been used in two small empirical studies to appraise the quality of evidence to support medical interventions in two medical areas [18,19]. While GRADE is applicable within the oral health field [20,21], there are no previous reports of an overall assessment of the quality of evidence in this area. Therefore, the aim of this study is to assess the quality of the evidence in contemporary oral health reviews using GRADE, and to assess possible differences in the quality of the evidence between Cochrane systematic reviews and non-Cochrane reviews.

Methods

The following pre-specified inclusion criteria were applied for the systematic reviews and meta-analyses:

Eligible systematic reviews or meta-analyses for inclusion in this study included at least one meta-analysis of at least two original studies; only one meta-analysis per systematic review was considered.

The selected meta-analysis was that reporting on the primary outcome or the first or most important reported outcome if the primary outcome was not specifically outlined.

If meta-analysis of more than one primary outcome was available, the meta-analysis including the largest number of trials was selected.

The following pre-specified exclusion criteria were applied:

- Systematic reviews with forest plots without a pooled estimate.
- Duplicate publications, laboratory studies and reviews of animal studies.
-Quantitative synthesis within single arms either due to absence of a control or due to analysis of before-after measurements.
- Reviews including meta-analysis, which included the same studies multiple times without explanation on whether subgroups were mutually exclusive.
- Systematic reviews or reviews using network meta-analysis.
- Diagnostic test accuracy reviews or reviews on prognostic factors.

Systematic reviews or meta-analyses published from January 2008 (year of GRADE adoption by the Cochrane Collaboration) until the end of 2013 were retrieved from the Oral Health Group (OHG) of the Cochrane Database of Systematic Reviews (CDSR) and by hand searching of 14 general and specialty dental journals (January 2008-December 2013) with the highest impact factor (IF) in 2012 (Table 2).

Download:

Table 2. Frequencies and percentages of included systematic reviews per journal type and journal impact factors.

https://doi.org/10.1371/journal.pone.0131644.t002

The titles and abstracts were initially read by one investigator (NP) and all full-text articles were retrieved and screened for inclusion. A second screening of the full reports was undertaken by one investigator (NP) resulting in exclusion of the reviews from further analyses based on the pre-specified exclusion criteria. Information was collected from all selected review articles at the review, meta-analysis and trial level (S1 Table).

The conclusions of selected meta-analyses were assessed using the GRADE approach by the first author, who was familiar with GRADE. A second author (PSF) verified the ratings; any disagreements were reconciled after discussion. The selected meta-analyses were assessed in relation to the quality of the evidence scored in the 5 domains specified within GRADE: Limitations in study design and/or execution (risk of bias) [12], inconsistency of results [22], indirectness of evidence [23], imprecision of results [24], and publication bias [25]. The overall GRADE rating results based on 4 levels (high, moderate, low and very low) are shown in Table 1. A more detailed description of the GRADE process is shown in the appendix (S1 File and S2 Table). The GRADE judgment for Cochrane reviews was re-assessed independently of the findings of the original review authors.

Data Analysis

The objectives of the statistical analyses were to tabulate frequency distributions of specific characteristics in the dental systematic review sample at the review, and meta-analysis levels together with the frequency distributions of the quality of the evidence in relation to GRADE for Cochrane systematic reviews and non-Cochrane reviews (or meta-analyses). Associations between systematic review/meta-analysis characteristics and GRADE assessment were also considered. The four-level GRADE rating (high, moderate, low, very low) was converted into a binary variable (high/moderate and low/very low) in order to fit a logistic regression model to assess potential associations between GRADE rating, impact factor and publication year. GRADE was the dependent variable and impact factor and publication year were the examined predictors. All the analyses were performed with Stata statistical software version 13.1 (Stata Corporation, College Station, Texas, USA).

Results

From the 510 reviews initially considered for inclusion in the study, 91 were included in the final assessment (Fig 1). The frequencies of reviews per journal are outlined in Table 2. Fifty reviews were included from the selected dental journals and 41 systematic reviews from The Cochrane Database for Systematic Reviews; a variety of conditions, interventions and outcomes were considered. The number of published systematic reviews with meta-analyses increased over time (S1 Fig). Significant differences were found between Cochrane and non-Cochrane reviews in respect of the region of authorship, involvement of a methodologist, number of collaborating centers, inclusion of GRADE assessment and assessment of harms in the outcome category. The Cochrane compared to non-Cochrane reviews are more likely to originate in Europe (OR: 14.62, 95% CI: 1.76, 121.19, p = 0.01) or Asia (OR: 1.43, 95% CI: 0.11, 18.00, p = 0.78) compared to Americas, to involve authors across multiple centers (OR: 5.26, 95% CI: 1.97, 14.09, p = 0.001), to include a methodologist (OR = 73.76, 95% CI:9.79, 555.93, p<0.001) and a GRADE assessment (OR = 12.71, 95% CI:4.17, 38.69, p<0.001), and to consider at least one harm in the outcomes (OR = 2.66, 95% CI: 1.13, 6.27, p = 0.03) (S3 Table).

Download:

Fig 1. Systematic review selection flow diagram.

https://doi.org/10.1371/journal.pone.0131644.g001

The Cochrane systematic reviews included only randomized clinical trials but the non-Cochrane reviews included both randomized and non-randomized studies and occasionally a combination of both designs. Authors of Cochrane reviews were more likely to account for clustering effects (OR: 15.62, 95% CI: 2.81, 86.76, p = 0.002) often encountered in dentistry either in the analysis or discussion and to correctly analyze paired data (OR: 11.25, 95% CI: 11.05, 541.20, p = 0.02) compared to non-Cochrane reviews (S4 Table).

A variety of approaches were used for the assessment of the methodological quality with the Cochrane risk of bias tool used in 62 out of the 91 reviews. In some reviews, a combination of approaches was used and in some instances incorrectly reporting guidelines were utilized for this purpose.

The GRADE assessment indicated that only 2% of the evidence belongs to the high and 18% to the moderate category with the remaining reviews at the low/very low level (Table 3). The distribution of the overall GRADE ratings and ratings for each GRADE domain were similar for Cochrane and non-Cochrane reviews (Table 3).

Download:

Table 3. Frequencies and percentages of overall and per domain GRADE rating for Cochrane and non-Cochrane meta-analyses.

https://doi.org/10.1371/journal.pone.0131644.t003

The most commonly downgraded domains were those for study limitations (risk of bias) and imprecision. The frequency of downgrading by 2 levels (very serious) was higher in the non-Cochrane reviews (46%) compared to the Cochrane reviews (22%); the former include also non-randomized studies. The inconsistency domain received similar ratings for non-Cochrane and Cochrane reviews in agreement with the reported statistical heterogeneity represented by I² (S2 Fig).

A GRADE assessment was included in 5/50 (10%) of the non-Cochrane and in 22/41 (54%) of the Cochrane reviews, respectively. Only three discrepancies were observed among the 27 included in the systematic reviews between the evidence rating found by the initial review authors and our re-analysis. One review published [26] in the Journal of Clinical Periodontology was based on a previous Cochrane systematic review which did not include a GRADE assessment [27]. Logistic regression analysis indicated that neither the journal impact factor (OR = 0.92, 95% CI: 0.68, 1.25, p = 0.61) nor the year of publication (OR: 1.17, 95% CI: 0.82, 1.67, p = 0.40) are important predictors for the quality of the evidence (Reference group: low/very low evidence).

Discussion

This cross-sectional meta-epidemiologic study is the first attempt to evaluate the evidence across the oral health field and has exposed that only 2% of the existing evidence derived from reviews has a high level of evidence and only 18% was of a moderate level of evidence in relation to primary outcomes with no difference between Cochrane and non-Cochrane reviews. This finding indicates that 80% of the evidence considered, much of which will form the basis of clinical recommendations, is at the low or very low level. This finding is alarming indicating the need to undertake high quality randomized clinical trials in order to reduce the uncertainty around therapies in the oral health field. Empirical studies in oral health and medicine in general using GRADE to assess the quality of evidence are sparse [18,19]. Our study included 91 reviews and covered the entire field of oral health systematic reviews and meta-analyses; it is likely that the quality of the evidence may vary across, between and within healthcare specialties. Although, GRADE has been used to assess the quality of evidence within specific systematic reviews in several fields, we were able to identify only two small empirical studies across the range of biomedical fields where GRADE was applied in meta-epidemiological studies on a body of systematic reviews to assess the quality of the evidence within a particular specialty. The scope of those studies was limited to hyperbaric oxygen therapy indications [19] and nonsurgical treatment of stress urinary incontinence [18]. In the analysis of hyperbaric oxygen treatment[19], the quality of the evidence ranged from very low to high depending on the indication with key modifiers identified as risk of bias, imprecision and large effect for observational studies. Similarities with our study include reasons for downgrading the quality of the evidence based on risk of bias and imprecision. The review on nonsurgical treatment of stress urinary incontinence involved assessment of 13 reviews finding that the quality of the evidence ranged from low to high depending on the intervention [18].

Cochrane reviews are considered to be particularly rigorous and are conducted according to strict criteria and usually include only randomized clinical trials for assessment of benefits and harms and only use quasi-randomized studies and observational studies for the assessment of harms [2]. In the present study, the non-Cochrane reviews had a relatively small proportion of non-randomized studies; however, the quality of evidence did not differ among the groups suggesting the quality of studies populating both Cochrane and non-Cochrane reviews to be lacking. The two most frequently downgraded domains were risk of bias (study limitations) and imprecision. For the risk of bias (study limitations) domain lack of allocation concealment, lack of blinding, losses to follow-up and selective reporting commonly led to a downgrade [28]. In relation to the imprecision domain, small sample size and wide confidence intervals which include both benefit and harm are likely to prompt a downgrade [24]. Specifically, for binary outcomes, if the 95% confidence intervals ranged from unimportant to important benefits and from unimportant to important harms, the quality of evidence was downgraded when appreciable benefit or harm are of the order of 25% relative risk reduction or increase. For continuous outcomes, if the 95% confidence intervals included no effect and upper and lower confidence interval bounds crossed the minimal important clinical difference (MID) for harm or benefit downgrade was considered appropriate. If the MID was not known scores were downgraded if the upper or lower confidence limit crossed an effect size of 0.5 in either direction [24]. Downgrade for these reasons, in particular, were common as the number and size of the included studies was usually small. This finding was consistent with a recent study evaluating the quality of evidence in hyperbaric oxygen therapy [19]. The latter review, however, was also based on a relatively limited subset of 17 reviews, with the majority of primary studies (75%) being non-randomized studies.

In relation to the characteristics of the reviews, Cochrane reviews were more likely to be published by European authors, to involve authors across multiple centers, to involve a methodologist, and to consider at least a single harm in the outcomes than was the case for non-Cochrane reviews. However, the strict methodological and reporting criteria and the involvement of a methodologist did not result in higher quality of evidence. This is logical as, while Cochrane reviews are likely to be conducted and reported to a higher standard, published systematic reviews are based on studies from the same pool, regardless of the effect of methodological quality of the review itself, review authorship and associated expertise. This finding does not suggest that Cochrane systematic reviews are redundant with a level of commensurate with non-Cochrane reviews; Cochrane reviews are rigorous and replicable and this should be the standard moving forward.

The systematic reviews included a median of 5 meta-analyses, each comprising of a median of 5 studies. These figures are in keeping with a large survey of the Cochrane Database of Systematic Reviews involving analysis of 2,321 systematic reviews which highlighted a median of 6 meta-analyses per review and a median of 3 studies per meta-analysis [29]. A similar previous review [30] of dental systematic reviews reported a median of 9 studies were included in the largest meta-analysis within dental systematic reviews involving 9 dental specialties, with the largest meta-analysis having no more than 4 studies in 19% of reviews. Similarly, the largest meta-analysis involved a median number of just two randomized clinical trials, although that review referred back to systematic reviews from as long ago as 1991 when randomized clinical trials were considerably less prevalent in oral health than is now the case.

In oral health, due to the fact that multiple matched or unmatched sites can receive the intervention of interest, clustering effects are common. If this is handled improperly during the analysis, significant results may arise which are not genuine [31–33]. Additionally, a common design which uses matching called split-mouth studies requires consideration of the within patient correlations during the synthesis. In split-mouth designs, clustering effects can also exist when both interventions are applied within the same patient and in the presence of multiple sites per treatment arm as is the case with teeth [33, 34]. Studies with clustering effects, paired data or a mixture of these were handled more appropriately in the Cochrane systematic reviews than in the non-Cochrane meta-analyses.

The Cochrane Collaboration adopted GRADE in 2008; this may explain the inclusion of a GRADE assessment more frequently in the Cochrane Database of Systematic Reviews with 22/41 (82%) Cochrane systematic reviews including a GRADE assessment versus 5/50 (10%) among non-Cochrane reviews. Therefore, while Cochrane systematic reviews follow strict methodology, it is evident that deviations from the pre-specified ideal are possible. We conducted our own assessment of GRADE highlighting a discrepancy in only 11% of occasions with the rating of the systematic review authors confirming the reliability of the GRADE approach when assessing the evidence [35]. Other investigators found larger inconsistencies during the application of GRADE [36], although the level of familiarity of these investigators with GRADE is unclear. In the present study one of the investigators who assessed the quality of the evidence had attended a GRADE course and both GRADE assessors have implemented GRADE previously in systematic reviews.

Older systematic reviews, time from search to publication and delays in systematic review updates may lead to important differences in included studies compared to more recent reviews [37]. Publication year, however, was not found to have a bearing on the quality of the evidence in the present study. Similarly, journal impact factor showed no association with the quality of the evidence. There is some evidence that systematic reviews published in higher impact medical journals are of higher methodological quality [38]; however, this does not necessarily translate into higher quality of evidence. Systematic review methodological quality should not be confused with the quality of the evidence, although more detailed and broader searches in particular may lead to the identification of more eligible studies and potentially lead to higher quality of evidence.

The present study did not include a cross-section of all oral health reviews but focused on the Cochrane Database of Systematic Reviews and dental journals with the highest impact factor, which may have potentially influenced the results, although the pool of potential candidate studies for inclusion is similar. Nevertheless, the observed findings are likely to represent a best-case scenario. The inclusion of the highest impact factor dental journals possibly explains the lack of association between impact factor and evidence quality. The several steps of the GRADE assessment may introduce an element of inconsistency in the ratings; it is, therefore, important that the complete picture is considered along with the judgment made. Rating of study limitations for non-randomized studies can be problematic especially when the information provided in the review is limited. A particularly challenging area to rate is publication bias as it is difficult to detect exclusion of eligible studies. Both the description of the literature search, reference to grey literature, trial registries and reference lists of included studies during the assessment of publication bias were considered to reach this decision. Statistical assessment of publication bias was typically impossible due to the small number of included studies in the meta-analyses [39]. It is of interest to note that the Agency for Healthcare Research and Quality Evidence based Practice Center Program considers publication bias to be an optional domain [40]. Finally, evaluation of imprecision can be often challenging. The GRADE approach suggests the use of confidence intervals since dichotomous decisions based on statistical significance are problematic [24]. Alternative approaches that can be used to decide upon the conclusiveness of the evidence have been proposed more recently [41].

In the present study, one author performed the initial screening of the eligible systematic reviews and one author made the GRADE rating with scores verified by a second author. In a previous study [36] implementation agreement ranged from low to high depending on the domain and concluded that both training in GRADE and clinical expertise are instrumental in improving consistency in its use. GRADE permits judgment in a methodical and transparent way [42] with inconsistencies relating to the type and number of outcomes considered during the assessment [36]. The approach favors an overall rating which may be considered as a continuum hinging on expertise and judgment resulting in a judgment where the overall may not be the sum of all parts [42]. Nevertheless, as an incidental finding, significant agreement was observed between the ratings made in the constituent reviews and in the present cross-sectional study. For the purposes of assessing the association between GRADE rating and publication year and impact factor, and because data was thin the 4-level scale was converted to two levels. We understand that GRADE uses 4 levels for a purpose and distilling the conclusions into a 2-level scale has limitations. In this instance, however, the 2-level presentation helped in communicating the results especially when applying GRADE for recommendations.

This study dealt with only one outcome for which meta-analyses were available. A more comprehensive approach could have involved consideration of several or all outcomes and could have included evidence from qualitative synthesis. However, this would have been very difficult to implement given the large number of systematic reviews. Furthermore, we feel that inclusion of only the primary or first outcome over a wide range of oral health systematic reviews is likely to be a good proxy of the quality of the evidence in the field of oral health research.

In conclusion, only a small proportion (20%) of the studies assessing interventions in oral health was of moderate (18%) or high (2%) quality according to GRADE. The most common domains provoking downgrading of the evidence were risk of bias (study limitations) and imprecision indicating the need for larger and higher quality randomized clinical trials to inform clinical decisions. The lack of robust evidence underpinning dental procedures should prompt a concerted drive to conduct funded, high quality clinical trials in order to improve the quality of the evidence for accepted but often unproven procedures.

Supporting Information

S1 Fig. Number of included non-Cochrane and Cochrane reviews by year of publications.

https://doi.org/10.1371/journal.pone.0131644.s001

(TIF)

S2 Fig. Heterogeneity (I-square %) for non-Cochrane and Cochrane included meta-analyses.

https://doi.org/10.1371/journal.pone.0131644.s002

(TIF)

S1 File. Detailed description of assessing the quality of the evidence per GRADE domain.

https://doi.org/10.1371/journal.pone.0131644.s003

(DOCX)

S1 Table. Collected information per systematic review at review, meta-analysis, trial level and GRADE assessment.

https://doi.org/10.1371/journal.pone.0131644.s004

(DOCX)

S2 Table. GRADE assessment at the selected meta-analysis level.

Mixed indicates inclusion of both randomized and non-randomised studies in the meta-analysis.

https://doi.org/10.1371/journal.pone.0131644.s005

(DOCX)

S3 Table. Frequencies and percentages of included review characteristics for Cochrane and non-Cochrane reviews at review level.

https://doi.org/10.1371/journal.pone.0131644.s006

(DOCX)

S4 Table. Frequencies and percentages of characteristics of selected meta-analyses for detailed assessment for 41 Cochrane and 50 not Cochrane reviews at meta-analysis level.

https://doi.org/10.1371/journal.pone.0131644.s007

(DOCX)

Acknowledgments

We would like to thank Ms Nancy Santesso from the GRADE group for helpful comments during the preparation of this manuscript.

Author Contributions

Conceived and designed the experiments: NP HW GS. Performed the experiments: NP PSF. Analyzed the data: NP GS. Contributed reagents/materials/analysis tools: NP PSF. Wrote the paper: NP PSF HW GS.

References

1. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Ann Intern Med. 2009;151: W65–94. pmid:19622512
- View Article
- PubMed/NCBI
- Google Scholar
2. Fleming PS, Seehra J, Polychronopoulou A, Fedorowicz Z, Pandis N. Cochrane and non-Cochrane systematic reviews in leading orthodontic journals: a quality paradigm? Eur J Orthod. 2013;35: 244–248. pmid:22510325
- View Article
- PubMed/NCBI
- Google Scholar
3. Seehra J, Fleming PS, Polychronopoulou A, Pandis N. Reporting completeness of abstracts of systematic reviews published in leading dental specialty journals. Eur J Oral Sci. 2013;121: 57–62. pmid:23489893
- View Article
- PubMed/NCBI
- Google Scholar
4. Kiriakou J, Pandis N, Fleming PS, Madianos P, Polychronopoulou A. Reporting quality of systematic review abstracts in leading oral implantology journals. J Dent. 2013;41: 1181–1187. pmid:24075952
- View Article
- PubMed/NCBI
- Google Scholar
5. Fleming PS, Seehra J, Polychronopoulou A, Fedorowicz Z, Pandis N. A PRISMA assessment of the reporting quality of systematic reviews in orthodontics. Angle Orthod. 2013;83: 158–163. pmid:22720835
- View Article
- PubMed/NCBI
- Google Scholar
6. Martel G, Duhaime S, Barkun JS, Boushey RP, Ramsay CR, Fergusson DA. The quality of research synthesis in surgery: the case of laparoscopic surgery for colorectal cancer. Syst Rev. 2012;1: 14. pmid:22588035
- View Article
- PubMed/NCBI
- Google Scholar
7. Brito JP, Tsapas A, Griebeler ML, Wang Z, Prutsky GJ, Domecq JP, et al. Systematic reviews supporting practice guideline recommendations lack protection against bias. J Clin Epidemiol. 2013;66: 633–638. pmid:23510557
- View Article
- PubMed/NCBI
- Google Scholar
8. Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336: 924–926. pmid:18436948
- View Article
- PubMed/NCBI
- Google Scholar
9. Atkins D, Eccles M, Flottorp S, Guyatt GH, Henry D, Hill S, et al. Systems for grading the quality of evidence and the strength of recommendations I: critical appraisal of existing approaches The GRADE Working Group. BMC Health Serv Res. 2004;4: 38. pmid:15615589
- View Article
- PubMed/NCBI
- Google Scholar
10. Guyatt GH, Oxman AD, Kunz R, Atkins D, Brozek J, Vist G, et al. GRADE guidelines: 2. Framing the question and deciding on important outcomes. J Clin Epidemiol. 2011;64: 395–400. pmid:21194891
- View Article
- PubMed/NCBI
- Google Scholar
11. Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011;64: 383–394. pmid:21195583
- View Article
- PubMed/NCBI
- Google Scholar
12. Balshem H, Helfand M, Schünemann HJ, Oxman AD, Kunz R, Brozek J, et al. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol. 2011;64: 401–406. pmid:21208779
- View Article
- PubMed/NCBI
- Google Scholar
13. http://www.gradeworkinggroup.org/society/index.htm [Internet].
14. De Palma R, Liberati A, Ciccone G, Bandieri E, Belfiglio M, Ceccarelli M, et al. Developing clinical recommendations for breast, colorectal, and lung cancer adjuvant treatments using the GRADE system: a study from the Programma Ricerca e Innovazione Emilia Romagna Oncology Research Group. J Clin Oncol Off J Am Soc Clin Oncol. 2008;26: 1033–1039.
- View Article
- Google Scholar
15. Swiglo BA, Murad MH, Schünemann HJ, Kunz R, Vigersky RA, Guyatt GH, et al. A case for clarity, consistency, and helpfulness: state-of-the-art clinical practice guidelines in endocrinology using the grading of recommendations, assessment, development, and evaluation system. J Clin Endocrinol Metab. 2008;93: 666–673. pmid:18171699
- View Article
- PubMed/NCBI
- Google Scholar
16. Guyatt G, Gutterman D, Baumann MH, Addrizzo-Harris D, Hylek EM, Phillips B, et al. Grading strength of recommendations and quality of evidence in clinical guidelines: report from an american college of chest physicians task force. Chest. 2006;129: 174–181. pmid:16424429
- View Article
- PubMed/NCBI
- Google Scholar
17. Brozek JL, Akl EA, Alonso-Coello P, Lang D, Jaeschke R, Williams JW, et al. Grading quality of evidence and strength of recommendations in clinical practice guidelines. Part 1 of 3. An overview of the GRADE approach and grading quality of evidence about interventions. Allergy. 2009;64: 669–677. pmid:19210357
- View Article
- PubMed/NCBI
- Google Scholar
18. Latthe P, Foon R, Khan K. Nonsurgical treatment of stress urinary incontinence (SUI): grading of evidence in systematic reviews. BJOG Int J Obstet Gynaecol. 2008;115: 435–444.
- View Article
- Google Scholar
19. Murad MH, Altayar O, Bennett M, Wei JC, Claus PL, Asi N, et al. Using GRADE for evaluating the quality of evidence in hyperbaric oxygen therapy clarifies evidence limitations. J Clin Epidemiol. 2014;67: 65–72. pmid:24189086
- View Article
- PubMed/NCBI
- Google Scholar
20. Faggion CM Jr. The shortened dental arch revisited: from evidence to recommendations by the use of the GRADE approach. J Oral Rehabil. 2011;38: 940–949. pmid:21707696
- View Article
- PubMed/NCBI
- Google Scholar
21. Faggion CM Jr. Grading the quality of evidence and the strength of recommendations in clinical dentistry: a critical review of 2 prominent approaches. J Evid-Based Dent Pract. 2010;10: 78–85. pmid:20466314
- View Article
- PubMed/NCBI
- Google Scholar
22. Guyatt GH, Oxman AD, Kunz R, Woodcock J, Brozek J, Helfand M, et al. GRADE guidelines: 7. Rating the quality of evidence—inconsistency. J Clin Epidemiol. 2011;64: 1294–1302. pmid:21803546
- View Article
- PubMed/NCBI
- Google Scholar
23. Guyatt GH, Oxman AD, Kunz R, Woodcock J, Brozek J, Helfand M, et al. GRADE guidelines: 8. Rating the quality of evidence—indirectness. J Clin Epidemiol. 2011;64: 1303–1310. pmid:21802903
- View Article
- PubMed/NCBI
- Google Scholar
24. Guyatt GH, Oxman AD, Kunz R, Brozek J, Alonso-Coello P, Rind D, et al. GRADE guidelines 6. Rating the quality of evidence—imprecision. J Clin Epidemiol. 2011;64: 1283–1293. pmid:21839614
- View Article
- PubMed/NCBI
- Google Scholar
25. Guyatt GH, Oxman AD, Montori V, Vist G, Kunz R, Brozek J, et al. GRADE guidelines: 5. Rating the quality of evidence—publication bias. J Clin Epidemiol. 2011;64: 1277–1282. pmid:21802904
- View Article
- PubMed/NCBI
- Google Scholar
26. Eberhard J, Jervøe-Storm P-M, Needleman I, Worthington H, Jepsen S. Full-mouth treatment concepts for chronic periodontitis: a systematic review. J Clin Periodontol. 2008;35: 591–604. pmid:18498383
- View Article
- PubMed/NCBI
- Google Scholar
27. Eberhard J, Jepsen S, Jervøe-Storm P-M, Needleman I, Worthington HV. Full-mouth disinfection for the treatment of adult chronic periodontitis. Cochrane Database Syst Rev. 2008; CD004622. https://doi.org/10.1002/14651858.CD004622.pub2
28. Guyatt GH, Oxman AD, Vist G, Kunz R, Brozek J, Alonso-Coello P, et al. GRADE guidelines: 4. Rating the quality of evidence—study limitations (risk of bias). J Clin Epidemiol. 2011;64: 407–415. pmid:21247734
- View Article
- PubMed/NCBI
- Google Scholar
29. Davey J, Turner RM, Clarke MJ, Higgins JPT. Characteristics of meta-analyses and their component studies in the Cochrane Database of Systematic Reviews: a cross-sectional, descriptive analysis. BMC Med Res Methodol. 2011;11: 160. pmid:22114982
- View Article
- PubMed/NCBI
- Google Scholar
30. Saltaji H, Cummings GG, Armijo-Olivo S, Major MP, Amin M, Major PW, et al. A descriptive analysis of oral health systematic reviews published 1991–2012: cross sectional study. PloS One. 2013;8: e74545. pmid:24098657
- View Article
- PubMed/NCBI
- Google Scholar
31. Fleming PS, Koletsi D, Polychronopoulou A, Eliades T, Pandis N. Are clustering effects accounted for in statistical analysis in leading dental specialty journals? J Dent. 2013;41: 265–270. pmid:23201411
- View Article
- PubMed/NCBI
- Google Scholar
32. Koletsi D, Pandis N, Polychronopoulou A, Eliades T. Does published orthodontic research account for clustering effects during statistical data analysis? Eur J Orthod. 2012;34: 287–292. pmid:22015822
- View Article
- PubMed/NCBI
- Google Scholar
33. Pandis N, Walsh T, Polychronopoulou A, Eliades T. Cluster randomized clinical trials in orthodontics: design, analysis and reporting issues. Eur J Orthod. 2013;35: 669–675. pmid:23041934
- View Article
- PubMed/NCBI
- Google Scholar
34. Pandis N, Walsh T, Polychronopoulou A, Katsaros C, Eliades T. Split-mouth designs in orthodontics: an overview with applications to orthodontic clinical trials. Eur J Orthod. 2013;35: 783–789. pmid:23376899
- View Article
- PubMed/NCBI
- Google Scholar
35. Mustafa RA, Santesso N, Brozek J, Akl EA, Walter SD, Norman G, et al. The GRADE approach is reproducible in assessing the quality of evidence of quantitative evidence syntheses. J Clin Epidemiol. 2013;66: 736–742.e5. pmid:23623694
- View Article
- PubMed/NCBI
- Google Scholar
36. Hartling L, Fernandes RM, Seida J, Vandermeer B, Dryden DM. From the Trenches: A Cross-Sectional Study Applying the GRADE Tool in Systematic Reviews of Healthcare Interventions. Malaga G, editor. PLoS ONE. 2012;7: e34697. pmid:22496843
- View Article
- PubMed/NCBI
- Google Scholar
37. Beller EM, Chen JK-H, Wang UL-H, Glasziou PP. Are systematic reviews up-to-date at the time of publication? Syst Rev. 2013;2: 36. pmid:23714302
- View Article
- PubMed/NCBI
- Google Scholar
38. Fleming PS, Koletsi D, Seehra J, Pandis N. Systematic reviews published in higher impact clinical journals were of higher quality. J Clin Epidemiol. 2014; https://doi.org/10.1016/j.jclinepi.2014.01.002
39. Sterne JAC, Egger M, Moher D (editors). Chapter 10: Addressing reporting biases. In: Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions. Version 5.0.1 [updated September 2008]. The Cochrane Collaboration, 2008. Available from www.cochrane-handbook.org.
40. Owens DK, Lohr KN, Atkins D, Treadwell JR, Reston JT, Bass EB, et al. AHRQ series paper 5: grading the strength of a body of evidence when comparing medical interventions—agency for healthcare research and quality and the effective health-care program. J Clin Epidemiol. 2010;63: 513–523. pmid:19595577
- View Article
- PubMed/NCBI
- Google Scholar
41. Ferreira ML, Herbert RD, Crowther MJ, Verhagen A, Sutton AJ. When is a further clinical trial justified? BMJ. 2012;345: e5913. pmid:22977141
- View Article
- PubMed/NCBI
- Google Scholar
42. Guyatt G, Oxman AD, Sultan S, Brozek J, Glasziou P, Alonso-Coello P, et al. GRADE guidelines: 11. Making an overall rating of confidence in effect estimates for a single outcome and for all outcomes. J Clin Epidemiol. 2013;66: 151–157. pmid:22542023
- View Article
- PubMed/NCBI
- Google Scholar

[ref1] 1. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Ann Intern Med. 2009;151: W65–94. pmid:19622512
View Article
PubMed/NCBI
Google Scholar

[2] View Article

[3] PubMed/NCBI

[4] Google Scholar

[ref2] 2. Fleming PS, Seehra J, Polychronopoulou A, Fedorowicz Z, Pandis N. Cochrane and non-Cochrane systematic reviews in leading orthodontic journals: a quality paradigm? Eur J Orthod. 2013;35: 244–248. pmid:22510325
View Article
PubMed/NCBI
Google Scholar

[6] View Article

[7] PubMed/NCBI

[8] Google Scholar

[ref3] 3. Seehra J, Fleming PS, Polychronopoulou A, Pandis N. Reporting completeness of abstracts of systematic reviews published in leading dental specialty journals. Eur J Oral Sci. 2013;121: 57–62. pmid:23489893
View Article
PubMed/NCBI
Google Scholar

[10] View Article

[11] PubMed/NCBI

[12] Google Scholar

[ref4] 4. Kiriakou J, Pandis N, Fleming PS, Madianos P, Polychronopoulou A. Reporting quality of systematic review abstracts in leading oral implantology journals. J Dent. 2013;41: 1181–1187. pmid:24075952
View Article
PubMed/NCBI
Google Scholar

[14] View Article

[15] PubMed/NCBI

[16] Google Scholar

[ref5] 5. Fleming PS, Seehra J, Polychronopoulou A, Fedorowicz Z, Pandis N. A PRISMA assessment of the reporting quality of systematic reviews in orthodontics. Angle Orthod. 2013;83: 158–163. pmid:22720835
View Article
PubMed/NCBI
Google Scholar

[18] View Article

[19] PubMed/NCBI

[20] Google Scholar

[ref6] 6. Martel G, Duhaime S, Barkun JS, Boushey RP, Ramsay CR, Fergusson DA. The quality of research synthesis in surgery: the case of laparoscopic surgery for colorectal cancer. Syst Rev. 2012;1: 14. pmid:22588035
View Article
PubMed/NCBI
Google Scholar

[22] View Article

[23] PubMed/NCBI

[24] Google Scholar

[ref7] 7. Brito JP, Tsapas A, Griebeler ML, Wang Z, Prutsky GJ, Domecq JP, et al. Systematic reviews supporting practice guideline recommendations lack protection against bias. J Clin Epidemiol. 2013;66: 633–638. pmid:23510557
View Article
PubMed/NCBI
Google Scholar

[26] View Article

[27] PubMed/NCBI

[28] Google Scholar

[ref8] 8. Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336: 924–926. pmid:18436948
View Article
PubMed/NCBI
Google Scholar

[30] View Article

[31] PubMed/NCBI

[32] Google Scholar

[ref9] 9. Atkins D, Eccles M, Flottorp S, Guyatt GH, Henry D, Hill S, et al. Systems for grading the quality of evidence and the strength of recommendations I: critical appraisal of existing approaches The GRADE Working Group. BMC Health Serv Res. 2004;4: 38. pmid:15615589
View Article
PubMed/NCBI
Google Scholar

[34] View Article

[35] PubMed/NCBI

[36] Google Scholar

[ref10] 10. Guyatt GH, Oxman AD, Kunz R, Atkins D, Brozek J, Vist G, et al. GRADE guidelines: 2. Framing the question and deciding on important outcomes. J Clin Epidemiol. 2011;64: 395–400. pmid:21194891
View Article
PubMed/NCBI
Google Scholar

[38] View Article

[39] PubMed/NCBI

[40] Google Scholar

[ref11] 11. Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011;64: 383–394. pmid:21195583
View Article
PubMed/NCBI
Google Scholar

[42] View Article

[43] PubMed/NCBI

[44] Google Scholar

[ref12] 12. Balshem H, Helfand M, Schünemann HJ, Oxman AD, Kunz R, Brozek J, et al. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol. 2011;64: 401–406. pmid:21208779
View Article
PubMed/NCBI
Google Scholar

[46] View Article

[47] PubMed/NCBI

[48] Google Scholar

[ref13] 13. http://www.gradeworkinggroup.org/society/index.htm [Internet].

[ref14] 14. De Palma R, Liberati A, Ciccone G, Bandieri E, Belfiglio M, Ceccarelli M, et al. Developing clinical recommendations for breast, colorectal, and lung cancer adjuvant treatments using the GRADE system: a study from the Programma Ricerca e Innovazione Emilia Romagna Oncology Research Group. J Clin Oncol Off J Am Soc Clin Oncol. 2008;26: 1033–1039.
View Article
Google Scholar

[51] View Article

[52] Google Scholar

[ref15] 15. Swiglo BA, Murad MH, Schünemann HJ, Kunz R, Vigersky RA, Guyatt GH, et al. A case for clarity, consistency, and helpfulness: state-of-the-art clinical practice guidelines in endocrinology using the grading of recommendations, assessment, development, and evaluation system. J Clin Endocrinol Metab. 2008;93: 666–673. pmid:18171699
View Article
PubMed/NCBI
Google Scholar

[54] View Article

[55] PubMed/NCBI

[56] Google Scholar

[ref16] 16. Guyatt G, Gutterman D, Baumann MH, Addrizzo-Harris D, Hylek EM, Phillips B, et al. Grading strength of recommendations and quality of evidence in clinical guidelines: report from an american college of chest physicians task force. Chest. 2006;129: 174–181. pmid:16424429
View Article
PubMed/NCBI
Google Scholar

[58] View Article

[59] PubMed/NCBI

[60] Google Scholar

[ref17] 17. Brozek JL, Akl EA, Alonso-Coello P, Lang D, Jaeschke R, Williams JW, et al. Grading quality of evidence and strength of recommendations in clinical practice guidelines. Part 1 of 3. An overview of the GRADE approach and grading quality of evidence about interventions. Allergy. 2009;64: 669–677. pmid:19210357
View Article
PubMed/NCBI
Google Scholar

[62] View Article

[63] PubMed/NCBI

[64] Google Scholar

[ref18] 18. Latthe P, Foon R, Khan K. Nonsurgical treatment of stress urinary incontinence (SUI): grading of evidence in systematic reviews. BJOG Int J Obstet Gynaecol. 2008;115: 435–444.
View Article
Google Scholar

[66] View Article

[67] Google Scholar

[ref19] 19. Murad MH, Altayar O, Bennett M, Wei JC, Claus PL, Asi N, et al. Using GRADE for evaluating the quality of evidence in hyperbaric oxygen therapy clarifies evidence limitations. J Clin Epidemiol. 2014;67: 65–72. pmid:24189086
View Article
PubMed/NCBI
Google Scholar

[69] View Article

[70] PubMed/NCBI

[71] Google Scholar

[ref20] 20. Faggion CM Jr. The shortened dental arch revisited: from evidence to recommendations by the use of the GRADE approach. J Oral Rehabil. 2011;38: 940–949. pmid:21707696
View Article
PubMed/NCBI
Google Scholar

[73] View Article

[74] PubMed/NCBI

[75] Google Scholar

[ref21] 21. Faggion CM Jr. Grading the quality of evidence and the strength of recommendations in clinical dentistry: a critical review of 2 prominent approaches. J Evid-Based Dent Pract. 2010;10: 78–85. pmid:20466314
View Article
PubMed/NCBI
Google Scholar

[77] View Article

[78] PubMed/NCBI

[79] Google Scholar

[ref22] 22. Guyatt GH, Oxman AD, Kunz R, Woodcock J, Brozek J, Helfand M, et al. GRADE guidelines: 7. Rating the quality of evidence—inconsistency. J Clin Epidemiol. 2011;64: 1294–1302. pmid:21803546
View Article
PubMed/NCBI
Google Scholar

[81] View Article

[82] PubMed/NCBI

[83] Google Scholar

[ref23] 23. Guyatt GH, Oxman AD, Kunz R, Woodcock J, Brozek J, Helfand M, et al. GRADE guidelines: 8. Rating the quality of evidence—indirectness. J Clin Epidemiol. 2011;64: 1303–1310. pmid:21802903
View Article
PubMed/NCBI
Google Scholar

[85] View Article

[86] PubMed/NCBI

[87] Google Scholar

[ref24] 24. Guyatt GH, Oxman AD, Kunz R, Brozek J, Alonso-Coello P, Rind D, et al. GRADE guidelines 6. Rating the quality of evidence—imprecision. J Clin Epidemiol. 2011;64: 1283–1293. pmid:21839614
View Article
PubMed/NCBI
Google Scholar

[89] View Article

[90] PubMed/NCBI

[91] Google Scholar

[ref25] 25. Guyatt GH, Oxman AD, Montori V, Vist G, Kunz R, Brozek J, et al. GRADE guidelines: 5. Rating the quality of evidence—publication bias. J Clin Epidemiol. 2011;64: 1277–1282. pmid:21802904
View Article
PubMed/NCBI
Google Scholar

[93] View Article

[94] PubMed/NCBI

[95] Google Scholar

[ref26] 26. Eberhard J, Jervøe-Storm P-M, Needleman I, Worthington H, Jepsen S. Full-mouth treatment concepts for chronic periodontitis: a systematic review. J Clin Periodontol. 2008;35: 591–604. pmid:18498383
View Article
PubMed/NCBI
Google Scholar

[97] View Article

[98] PubMed/NCBI

[99] Google Scholar

[ref27] 27. Eberhard J, Jepsen S, Jervøe-Storm P-M, Needleman I, Worthington HV. Full-mouth disinfection for the treatment of adult chronic periodontitis. Cochrane Database Syst Rev. 2008; CD004622. https://doi.org/10.1002/14651858.CD004622.pub2

[ref28] 28. Guyatt GH, Oxman AD, Vist G, Kunz R, Brozek J, Alonso-Coello P, et al. GRADE guidelines: 4. Rating the quality of evidence—study limitations (risk of bias). J Clin Epidemiol. 2011;64: 407–415. pmid:21247734
View Article
PubMed/NCBI
Google Scholar

[102] View Article

[103] PubMed/NCBI

[104] Google Scholar

[ref29] 29. Davey J, Turner RM, Clarke MJ, Higgins JPT. Characteristics of meta-analyses and their component studies in the Cochrane Database of Systematic Reviews: a cross-sectional, descriptive analysis. BMC Med Res Methodol. 2011;11: 160. pmid:22114982
View Article
PubMed/NCBI
Google Scholar

[106] View Article

[107] PubMed/NCBI

[108] Google Scholar

[ref30] 30. Saltaji H, Cummings GG, Armijo-Olivo S, Major MP, Amin M, Major PW, et al. A descriptive analysis of oral health systematic reviews published 1991–2012: cross sectional study. PloS One. 2013;8: e74545. pmid:24098657
View Article
PubMed/NCBI
Google Scholar

[110] View Article

[111] PubMed/NCBI

[112] Google Scholar

[ref31] 31. Fleming PS, Koletsi D, Polychronopoulou A, Eliades T, Pandis N. Are clustering effects accounted for in statistical analysis in leading dental specialty journals? J Dent. 2013;41: 265–270. pmid:23201411
View Article
PubMed/NCBI
Google Scholar

[114] View Article

[115] PubMed/NCBI

[116] Google Scholar

[ref32] 32. Koletsi D, Pandis N, Polychronopoulou A, Eliades T. Does published orthodontic research account for clustering effects during statistical data analysis? Eur J Orthod. 2012;34: 287–292. pmid:22015822
View Article
PubMed/NCBI
Google Scholar

[118] View Article

[119] PubMed/NCBI

[120] Google Scholar

[ref33] 33. Pandis N, Walsh T, Polychronopoulou A, Eliades T. Cluster randomized clinical trials in orthodontics: design, analysis and reporting issues. Eur J Orthod. 2013;35: 669–675. pmid:23041934
View Article
PubMed/NCBI
Google Scholar

[122] View Article

[123] PubMed/NCBI

[124] Google Scholar

[ref34] 34. Pandis N, Walsh T, Polychronopoulou A, Katsaros C, Eliades T. Split-mouth designs in orthodontics: an overview with applications to orthodontic clinical trials. Eur J Orthod. 2013;35: 783–789. pmid:23376899
View Article
PubMed/NCBI
Google Scholar

[126] View Article

[127] PubMed/NCBI

[128] Google Scholar

[ref35] 35. Mustafa RA, Santesso N, Brozek J, Akl EA, Walter SD, Norman G, et al. The GRADE approach is reproducible in assessing the quality of evidence of quantitative evidence syntheses. J Clin Epidemiol. 2013;66: 736–742.e5. pmid:23623694
View Article
PubMed/NCBI
Google Scholar

[130] View Article

[131] PubMed/NCBI

[132] Google Scholar

[ref36] 36. Hartling L, Fernandes RM, Seida J, Vandermeer B, Dryden DM. From the Trenches: A Cross-Sectional Study Applying the GRADE Tool in Systematic Reviews of Healthcare Interventions. Malaga G, editor. PLoS ONE. 2012;7: e34697. pmid:22496843
View Article
PubMed/NCBI
Google Scholar

[134] View Article

[135] PubMed/NCBI

[136] Google Scholar

[ref37] 37. Beller EM, Chen JK-H, Wang UL-H, Glasziou PP. Are systematic reviews up-to-date at the time of publication? Syst Rev. 2013;2: 36. pmid:23714302
View Article
PubMed/NCBI
Google Scholar

[138] View Article

[139] PubMed/NCBI

[140] Google Scholar

[ref38] 38. Fleming PS, Koletsi D, Seehra J, Pandis N. Systematic reviews published in higher impact clinical journals were of higher quality. J Clin Epidemiol. 2014; https://doi.org/10.1016/j.jclinepi.2014.01.002

[ref39] 39. Sterne JAC, Egger M, Moher D (editors). Chapter 10: Addressing reporting biases. In: Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions. Version 5.0.1 [updated September 2008]. The Cochrane Collaboration, 2008. Available from www.cochrane-handbook.org.

[ref40] 40. Owens DK, Lohr KN, Atkins D, Treadwell JR, Reston JT, Bass EB, et al. AHRQ series paper 5: grading the strength of a body of evidence when comparing medical interventions—agency for healthcare research and quality and the effective health-care program. J Clin Epidemiol. 2010;63: 513–523. pmid:19595577
View Article
PubMed/NCBI
Google Scholar

[144] View Article

[145] PubMed/NCBI

[146] Google Scholar

[ref41] 41. Ferreira ML, Herbert RD, Crowther MJ, Verhagen A, Sutton AJ. When is a further clinical trial justified? BMJ. 2012;345: e5913. pmid:22977141
View Article
PubMed/NCBI
Google Scholar

[148] View Article

[149] PubMed/NCBI

[150] Google Scholar

[ref42] 42. Guyatt G, Oxman AD, Sultan S, Brozek J, Glasziou P, Alonso-Coello P, et al. GRADE guidelines: 11. Making an overall rating of confidence in effect estimates for a single outcome and for all outcomes. J Clin Epidemiol. 2013;66: 151–157. pmid:22542023
View Article
PubMed/NCBI
Google Scholar

[152] View Article

[153] PubMed/NCBI

[154] Google Scholar

Figures

Abstract

Objectives

Study Design and Setting

Results

Conclusion

Introduction

Methods

Data Analysis

Results

Discussion

Supporting Information

S1 Fig. Number of included non-Cochrane and Cochrane reviews by year of publications.

S2 Fig. Heterogeneity (I-square %) for non-Cochrane and Cochrane included meta-analyses.

S1 File. Detailed description of assessing the quality of the evidence per GRADE domain.

S1 Table. Collected information per systematic review at review, meta-analysis, trial level and GRADE assessment.

S2 Table. GRADE assessment at the selected meta-analysis level.

S3 Table. Frequencies and percentages of included review characteristics for Cochrane and non-Cochrane reviews at review level.

S4 Table. Frequencies and percentages of characteristics of selected meta-analyses for detailed assessment for 41 Cochrane and 50 not Cochrane reviews at meta-analysis level.

Acknowledgments

Author Contributions

References