Introduction

Much of the global burden of disease requires surgical intervention and over 234 million operations are conducted each year worldwide [1, 2]. Lack of timely access to high-quality surgical care remains a major problem in much of the world, even though surgical interventions can be cost-effective interventions in terms of lives saved and disabilities avoided [3]. In addition, perioperative period is frequently associated with morbidity and mortality [4]; in high-income countries, major complications are occurring in 3 to 16% of in-hospital surgeries leading to permanent disability or mortality ranging from 0.4 to 0.8%, while increased to 5% to 10% in low- and middle-income countries [5].

Clinical practice guidelines (CPG) are statements containing evidence-based recommendations aimed at providing better patient care and helping physicians and patients to make the best decisions for the prevention, diagnosis, and treatment of different diseases [6]. The use of CPG during preoperative assessment and treatment of patients before surgical procedures can contribute to reducing the risk of complications, increase patient safety, improve the quality of care, enable the implementation of effective interventions, decrease treatment variability, and finally improve patient outcomes [7].

The development of high-quality CPG is a complex, lengthy, and systematic scientific process involving developers, stakeholders, and users, and therefore, the development process and produced recommendations can present varying degrees of quality [6, 8]. CPG have been evaluated by several authors including numerous conditions and years of development and publication (from 1980 to 2019). In general, the applicability domain has received the lowest mean score, and improvement in time is still a matter of controversy [9–12].

On perioperative care, Barajas-Nava et al., evaluated 22 CPG from 1990 up to 2008 describing their quality as moderate for most of the domains with the lowest scores in stakeholder involvement, applicability and editorial independence [7]. Ciapponi et al. reported the domain “applicability” as the worst score in preoperative (2010 to 2017) CPG [13]. In addition, a recent evaluation of guidelines in airway management, as part of perioperative care, showed applicability scores of around 23% [14]. Studies were not able to detect any improvement over time in CPG quality, specifically in this context [7, 14], and show opportunities to improve the quality of CPG development [13].

We aimed to critically evaluate the transparency and methodological quality of the development of published CPG in the preoperative assessment and management of adult patients undergoing elective surgery published in the last decade, by using the AGREE-II tool.

Methods

Results

We included 20 CPG in our analysis (Fig 1). During our eligibility process, we excluded 14 CPG, text in S3 Appendix. Based on the mean quality score, 14 CPG (70%) were classified as “recommended” to be implemented in clinical practice (Tables 1 and 2). A total of 16 CPG (80%) were produced by medical societies, 12 (60%) were new guidelines (first version), and only 10 (50%) explicitly reported the source of funding for development. Most CPG (95%) described they used a systematic review process and only six (30%) reported a formal method for achieving consensus.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Search and selection process of CPG.

https://doi.org/10.1371/journal.pone.0272756.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Characteristics of included CPG stratified by recommendation status* (n = 20).

https://doi.org/10.1371/journal.pone.0272756.t001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Scores of the AGREE-II domains for 20 included CPG stratified by category and recommendation status^*.

https://doi.org/10.1371/journal.pone.0272756.t002

Considering all CPG, the highest AGREE-II median scores were observed in the domains “scope and objective” (93%), “clarity of presentation” (84%), and “editorial independence” (97%). The lowest median scores were assigned to “stakeholder involvement” and “applicability” (56% and 44%, respectively) (Fig 2, Table 2). Overall, CPG focused on the use of routine preoperative tests were rated higher than CPG related to cardiac assessment for non-cardiac surgery and CPG related to preoperative fasting (81% versus 73% and 74% respectively).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Median scores of CPG evaluated in six domains of the AGREE-II instrument stratified by category (n = 20).

https://doi.org/10.1371/journal.pone.0272756.g002

The highest mean rated CPG among the preoperative fasting category was the ASA 2017 (93%) [19]; among the cardiac evaluation for non-cardiac surgery CPG were CCS 2017 (91%) [20], ESC-ESA 2014 (90%) [21], and AHA-ACC 2014 (89%) [22]. Finally, ICSI 2020 [23] was the best-rated CPG in the preoperative testing category (97%) (Table 2). Inter-rater agreement was rated as good to excellent for all domains (Table 3).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Inter-rater agreement of AGREE II domains.

https://doi.org/10.1371/journal.pone.0272756.t003

Mean AGREE-II scores among the included CPG ranged from 37% to 97% without major changes in the mean CPG scores over time (Table 2). Among the recommended CPG (n = 14, 62% to 97%), 10 reported funding, 2 were produced by governmental institutions, 10 by medical societies, and 2 by universities or academic institutions. In contrast, none of the CPG recommended with modifications reported funding, none were developed by governmental or academic institutions, and all were produced by medical societies. Overall, the most frequent characteristics presented in the recommended CPGs were being produced by governmental institutions (2 of 2, 100%), involving more than 20 authors (6 of 8, 75%), being the first version (9 of 12, 75%), reporting funding (10 of 10, 100%), and being developed by following a systematic review process (14 of 19, 73%) (Table 1). The number of included guidelines in this study prevents a more detailed analysis of associated factors.

Discussion

This systematic review included 20 CPG related to preoperative assessment and management of adult patients undergoing elective surgery. In the cardiac evaluation for non-cardiac surgery guidelines, two CPG (CCS 2017 [20] and ESC-ESA 2014 [21]) obtained AGREE-II scores equal or above 90%; In preoperative fasting, one CPG (ASA 2017 [19]) obtained a score of 93%, and in preoperative testing, one CPG (ICSI 2020 [23]) obtained a score of 97%. In general, 14 of 20 included CPG were classified as recommended to be used in clinical practice.

The domain “applicability” had the lowest scores; this has also been found extensively in other related studies [9–12, 38]. On perioperative care and airway management, applicability has been rated low [7, 13, 14]. Considering CPG are designed to be widely used, these findings represent important challenges to the guideline’s development and implementation. Many of the current CPG do not consider the inclusion of tools and strategies that facilitate their application in real clinical scenarios or barriers that limit their use. Sometimes, CPG do not provide insights on how the recommendations can be put into practice or regarding the resource consequences of applying them (i.e., in limited resources settings). That can be translated into a low score in this domain. In addition, the opinion of patients or patients’ representatives is rarely described in the development of recommendations resulting in low scores found in the domain “stakeholder involvement”.

Recent developments of CPG may be reflected by a continuous increase in some domains [39]. The domain "scope and objective" received a high score, showing that most included guidelines have a clear objective, a well-defined population under consideration, and some specific focused clinical questions. In the perioperative scenario, this remains a high-rated domain without major differences from previous assessments [7, 13].

Editorial independence is critical in the process of developing high-quality CPG [40] and it has been previously reported as one of the lowest scored domains [7]. Results of this study show that in the preoperative assessment and treatment of adult patients before elective surgery, most available guidelines of the last decade can be considered independent, in line with a recent assessment [13]. Across domains, editorial independence obtained the highest score (97%) but there are some CPG scoring zero in this domain. While this finding adds confidence to the provided recommendations by reducing the risk of delivering biased recommendations or being influenced by sponsors (seeking their benefits) or by the pharmaceutical industry, there are still some CPG produced without a clear and transparent editorial independence. This large scoring variability also reflects the relative importance each sponsor gives to the editorial independence process during development, and the great implications to reviewers and evaluators during judging this domain. Most of the included CPG were produced by specialty societies but specific conflicts of interest were declared as well as their potential effect on the development process and content of the guideline with transparency. All CPG classified as recommended reported their funding sources while none of the recommended with modifications did.

Methodological procedures used to generate recommendations are critical steps outlined by AGREE-II as the “rigour of development” domain. In comparisons to previous studies [9] and other related perioperative scenarios [14], included guidelines presented a median score of 67%, with higher scores for CPG produced in Europe, United States, and Canada. From 1980 to 2007, a very slow increase in quality in CPG was reported [9] However, in the last decade, the AGREE-II tool has become the most widely used international “gold standard” for guidelines development [41], potentially supporting the improvement in this area.

Recently Ciapponi et al., published a systematic review focused to summarize and compare recommendations presented in evidence-based CPG for preoperative care [13]. Including 16 CPG, they reported many strong recommendations ready to be considered for implementation. In line with these findings, our results classified most included CPG as “recommended”. In addition, when comparing our findings, they do not differ to a greater extent, being consistent that the domains top-scored were “scope and objective”, "Clarity of presentation", and "editorial independence". Also, “applicability” was the lowest rated.

Low applicability can be explained by the difference in the populations to which the intervention is directed or by the lack of technological resources to perform such intervention in different settings. In addition, as stated by Ciapponi et al., an inadequate adoption of guidelines (due to the tendency of many physicians to practice a "defensive medicine"), results in unnecessary interventions only to avoid legal problems and concerns derived from the care [13]. This is especially important for the use of routine preoperative tests.

There are three major barriers to CPG implementation described in the literature. Personal factors (related to physicians’ knowledge and attitudes), guideline-related factors, and external factors [42]. In addition, central elements of successful strategies for CPG implementation include dissemination, education and training, social interaction, decision support systems, and standing orders. A whole adjusted and adapted process is needed in advance when surgical teams are trying to implement a new guideline. Recent evidence indicates that only by following a structured process that includes an analysis of the potential local-related barriers, implementation and adherence can be effective [42].

This study did have some limitations. First, AGREE-II items do not include a category “does not apply”. In some cases, information to evaluate a single domain is missing and doubt remains as to how this should be scored and rated. Second, the degree of training required to allow an adequate and valid evaluation of a CPG is always a matter of concern, and certain subjective judgments may be present between evaluators. In our case, a detailed training process was implemented for the three evaluators regarding the application of the AGREE II tool, resulting in an agreement rated from good to excellent for all included areas. Third, to include updated versions of certain CPG may be controversial but it also allows us to compare the change in score with the previous versions. Finally, while most CPG are published in English, we included only CPG published in English and Spanish losing some relevant information from particular countries.

Healthcare providers use CPG considering guidelines to provide recommendations based on the best available evidence [43]. However, a 2009 study reported that most of the evidence used in the American College of Cardiology / American Heart Association CPG was based on expert recommendations [44], commonly non scientifically validated opinions. A systematic review of the evidence-based CPG published by the American and European scientific societies in anaesthesiology has recently been published. Their authors evaluated the quality of the evidence used to provide recommendations, finding that only 16% of them were based on level A of evidence, 33% on level B of evidence, and 51% on level C of evidence. These findings imply a critical need for greater efforts to improve the evidence used in the future CPG [43]. In our study, the limited number of CPG prevents a more detailed analysis of the factors associated with high methodological quality; further studies to assess the relationship between characteristics of CPG developers and the quality of CPG are warranted as they are available in other scenarios [45].

In 2022, O’Shaughnessy et al., assessed the quality of CPG published during the last 5 years in top anesthesia journals. With a scope beyond the preoperative care, they included 51 CPG with low scores to “stakeholder involvement”, “rigor of development” and “applicability” domains [46] Additionally, Mai et al. in 2021, evaluated 96 CPG in anesthesiology practice finding “rigor of development” and “applicability” as the lowest rated domains [47]. Publication of a CPG in a peer-reviewed high-quality journal may enhances the scientific credentials of the process [48] while reducing the potential inclusion CPG produced by low- and middle-income countries or in any language different from English. In addition, time barriers, peer review and the overall editorial process may distort and delay the original message and recommendations as part of the development of the guideline [49, 50].

In conclusion, CPG in the preoperative assessment or management of adult patients undergoing elective surgery including preoperative fasting, cardiac assessment for non-cardiac surgery, and use of routine preoperative tests present moderate to high methodological quality and can be recommended for their use or adaptation. Domains of applicability and stakeholder involvement must be improved in the development of future guidelines.

Supporting information

Acknowledgments

Assistance with the study: The authors are grateful to the Medical Library, Erasmus University Medical Centre Rotterdam for assistance with the literature search.

Presentation: Preliminary data for this study were presented as a poster presentation at the Colombian Society of Anesthesiology meeting, 5–7 August 2021.