Assessment of early goal-directed therapy guideline adherence: Balancing clinical importance and feasibility

Background Assessing adherence to Early goal-directed therapy (EGDT) is challenging and might account for the negative findings and generalisability of the major trials to a real-life setting. This study was aimed (1) to extract key components of pediatric EGDT guidelines potentially becoming adherence criteria; (2) to classify adherence criteria into complete, clinically important, and feasible; and (3) to compare percent adherence to selected guidelines using the three approaches. Methods This study started with review of existing evidence to extract key components of pediatric EGDT guidelines. Modified Delphi method was then conducted in two rounds among national experts to identify feasible and/or clinically important criteria. Data from the national prospective multicenter study “Clinical Effectiveness of the Utilization of Bundled Care for Severe Sepsis and Septicemia Children” at King Chulalongkorn Memorial Hospital (KCMH) during 1 June 2012 and 28 February 2014 was used to compare percentage of adherence across the three approaches. Results Of 28 components extracted from the review, 10 were identified by the national experts through the Modified Delphi as feasible whereas 8 were identified as clinically important. Thirty-one severe sepsis patients (48.39% male, median age 3.4 years) were reviewed. Sepsis mortality was 9.7%, a significant reduction from 19% and 42% in 2010 and 2007, respectively. Based on the complete adherence criteria, the percent adherence varied from 60.71% to 89.29% (overall mean 76.84%), with lower adherence in the dead than the survived cases (73.81% vs 77.17%; p = 0.55). The percent adherence varied by criteria used: 69.35%, 76.84%, and 84.52% for clinical importance, complete, and feasibility criteria, respectively. Conclusion Adherence determination based on selected clinical importance alone might result in an incorrectly estimated clinical benefit of EGDT guidelines, especially in a resource-limited setting. Both clinical importance and feasibility should be integrated into the development of adherence assessment criteria.


Results
Of 28 components extracted from the review, 10 were identified by the national experts through the Modified Delphi as feasible whereas 8 were identified as clinically important. Thirty-one severe sepsis patients (48.39% male, median age 3.4 years) were reviewed. Sepsis mortality was 9.7%, a significant reduction from 19% and 42% in 2010 and 2007, respectively. Based on the complete adherence criteria, the percent adherence varied from 60.71% to 89.29% (overall mean 76.84%), with lower adherence in the dead than the survived cases (73.81% vs 77.17%; p = 0.55). The percent adherence varied by criteria used: 69.35%, 76.84%, and 84.52% for clinical importance, complete, and feasibility criteria, respectively. PLOS

Introduction
Severe sepsis and septic shock are associated with significant morbidity and mortality [1,2]. Early goal-directed therapy (EGDT)-a strategy for hemodynamic optimization-has been advocated in clinical practice for sepsis management [2][3][4][5][6][7]. The Surviving Sepsis Campaign was introduced in 2004 and later updated in 2008 and 2012, covering management of adult and pediatric patients with severe sepsis and septic shock [8,9]. Unfortunately, many recommendations were based on low-quality adult-focused evidence and expert consensus [10]. Nonetheless, the guideline has been underused; for instance, only 7% of the emergency physicians routinely implemented adult EGDT, based on a survey of 30 academic tertiary care hospitals in the US [11]. Barriers to guideline implementation included a critical shortage of nursing staff, problems in obtaining central venous pressure (CVP) monitoring, and challenges in early identification of patients with sepsis [12,13]. The other factors were the lack of adequate time and staff, physical space in the emergency department (ED), communication with medical specialties, and identifying appropriate patients. The negative findings of the ProCESS trial [14], the ARISE study [15], as well as the ProM-ISe study [16] on the effectiveness of EGDT may not fully support the implementation of a standardised protocol for dynamic care process for adult patients. However, it should be noted that protocol-based care is not just a collection of clinically effective components but a series of inter-related interventions. Although assessing adherence to a complex guideline is challenging, these studies included only cases with full adherence to the guideline defined based on some unempirical adherence criteria.
As high-quality studies usually were conducted in a controlled setting with a full resource, the generalisability of the findings to less feasible settings was unquestionably limited. While information on the protocol adherence in the three major trials was presented in the supplementary appendix of the published articles, some significant discrepancies of compliances between both trials were noted but the relevant discussion was limited [14][15][16]. It would be unfair for institutions in resource-limited areas that could have benefited from 'partial' implementation of the guideline if only the findings from the major studies were used. In the real-life setting, both clinical importance and feasibility of the guideline components should be addressed.
Variation of adherence determination across studies exists whereas the three recent trials contained similar components (Table 1). For example, Mikkelsen et al used only ScvO2 measurement and achievement of hemodynamic goals as main outcome variables to identify factors associated with not initiating EGDT in emergency department [17]. Another study by Crowe et al introduced an extensive list of adherence criteria [18] to assess the effect of EGDT on mortality. Regardless of the negative findings from the modified version, the EGDT for sepsis has still been widely accepted.
Implementation feasibility of each guideline component was first addressed by O'Neill et al [19]. Seven core components of adult EGDT guideline were assessed for implementation difficulty in a community-based setting. Using percent adherence as the measure, arterial line placement, ScvO2 measurement were the most difficult elements.
Consequently, it was not uncommon to see a "simplified" or "reduced" version of EGDT be implemented in many resource-limited institutions whereas evidence on whether such approach offered comparable outcomes has been limited, especially in pediatric patients. Finding from a prospective cohort study by Sankar et al suggested that even intermittent (rather than continuous) ScvO2 monitoring could still reduce the mortality rate and improve organ dysfunction [20], which was later supported by the recent meta-analysis [21].
It could be assumed that ScvO2 monitoring was one of the critical EGDT components that could be modified when feasibility is of concern. However, whether this concept can be applied to the other EGDT components is still unclear. Empirical evidence on assessing the balance between clinical importance and feasibility of each EGDT component is essential. We synthesized adherence criteria from the pediatric EGDT sepsis guidelines, determined the feasibility of each guideline components, and comparatively assessed the guideline adherence of the care provided to the patients at our institution.

Synthesis of adherence criteria
A panel of five pediatric critical care experts was asked to discuss the EGDT guideline of the Surviving Sepsis Campaign 2012 [9] and selected literature on guideline compliance [14][15][16][17][18][19]. Finally, a list of key components was produced. EGDT components that were conceptual and/ or can be implemented at any time point were classified as 'general' whereas the other components were 'sequential' components.

Feasibility determination
A modified Delphi technique was used to determine the feasibility of each guideline component. Feasibility was defined as the likelihood of actual implementation perceived by the clinicians of a specific setting. This can be affected by organizational factors such as the availability of essential equipment and healthcare professional factors such as inadequate skill for central line insertion. Then, two rounds of the survey were conducted among 20 pediatric pulmonary & critical care specialists. They were regarded as clinical experts in the treatment of severe sepsis and septic shock in children and therefore should be able to identify the most feasible components of the EGDT sepsis guideline. Their responses in the first round were analysed in order to come up with an anonymous summary, which was presented in the second round of the survey.
The questionnaire contained the 28 items identified by the experts as described above. Each respondent was asked to respond to the question "Do you agree that each following items comply in your center?," using 5-point Likert scale (1 strongly disagree; 2 disagree; 3 neither agree or disagree; 4 agree; 5 strongly agree).

Exploratory EGDT adherence assessment at an institution
We defined the ten most agreeable components as feasible and all eight components of Pro-CESS, ARISE, and ProMISe studies as clinically important. Medical records of 31 randomly selected cases (one case per week for every other week) of severe sepsis and septic shock children at King Chulalongkorn Memorial Hospital (KCMH) during 1 June 2012 to 28 February 2014 were reviewed to assess the adherence to EGDT guideline based on the complete, feasibility, and clinical importance criteria.
This study was part of the "Clinical effectiveness of the utilization of bundled care for severe sepsis and septic shock in children at King Chulalongkorn Memorial Hospital" project, which was approved by Journal Research Ethics Committes (JREC), Bangkok, Thailand (JREC 015/55).

Synthesis of adherence criteria
The 28 components were divided into 18 general and 10 sequential components ( Table 2). All the extracted components from EGDT guideline were listed with the agreement score (mean +/-SD) in the modified Delphi rounds in S1 Table. The changes of means across the two rounds of Modified Delphi technique reflected the extent to which their respondent groups modified their responses to each of the factors whereas final interpretation was made using in the second round.

Exploratory EGDT adherence assessment at an institution
There were 15 male and 16 female patients with median age of 3.4 years (Interquartile Range: 0.7 to 11.8). Twenty-five patients survived, resulting in 9.7% mortality.
Adherence to all 28 items in each case at our institution was shown in S2 Table. For each of the patient, the adhered components were shaded. Percent adherence was calculated by dividing the number of shaded cells by the number of total cells. The row was sorted by percent adherence, from lowest to highest, with the three dead patients listed at the top. None of the patients had perfect adherence. The percent adherence varied from 60.71% to 89.29% (overall mean 76.84%), with insignificantly lower adherence in the dead than the survived cases (73.81% vs 77.17%; p = 0.5514).

Discussion
To our knowledge, our work is the first attempt that used a systematic approach to explore details of complex guideline for adherence in children with sepsis. The panel of experts helped to classify the 28 key components, extracted from the literature review, into feasible and clinically important ones. Given the non-representativeness of our experts, these components were used only for the exploration of adherence assessment in this study. Percent adherence assessed by various criteria was calculated, based on the data from a tertiary hospital in Thailand. Findings from our study suggested potential impact of criteria on the assessment of EGDT adherence. Also, even partial adherence to EGDT guideline might still contribute to better outcomes, regardless of the criteria used.
We explored the feasibility of many more EGDT components than the previous study by O'Neill et al (28 vs 7 components). In their study, arterial line insertion, CVP measurement, and ScvO2 monitoring were the least adhered components (42%, 27%, and 15% adherence, respectively [19]. Our study revealed similar findings (45% adherence to arterial line insertion) whereas higher 84% adherence to CVP measurement and ScvO2 monitoring at our facility might reflect the effect of setting (ED vs ICU).
Despite negative effect of EGDT on mortality in the major trials [14][15][16], a significant reduction of mortality at our site from 42% in 2007 to 19% in 2010 (p<0.02) [22] and 9.7% in this study still supported the clinical benefit of EGDT, even when partially implemented. However, many other factors could contribute to the reduction of mortality so the direct clinical benefit of EGST could not be inferred from our data. Based on the adherence criteria used by ProCESS, ARISE, and ProMISe studies, 94% (29/31) of our patients would have been excluded, suggesting that including only fully adhered cases might lead to potentially incorrect estimation of EGDT effect on clinical outcomes in the real-life setting.
The incremental gain from EGDT implementation might be relatively small in fully equipped settings with a high standard of care, as suggested by negative findings from the major trials [14][15][16]. One limitation of these well-designed clinical trials was the exclusion of partially adhered cases. However, findings from our study suggested that a partially implemented EGDT might still improve clinical outcomes, at least in a developing country like Thailand. To deal with imperfect adherence in real life setting, a balanced determination between clinically important and feasible components must be carefully considered using prioritization approaches presented in Table 4. In the ideal setting, all the 28 components must be fully implemented. Otherwise, the two clinically important and feasible components (G5 and S2) must be done. Next, barriers to implementing the other six clinically important components (G8, G9, G11, G15, S9, S10) should be removed. When all clinically important components are fully adhered, the eight feasible-yet-less important components (G1, G2, G3, G4, G6, S1, S7, S8) are encouraged. As feasibility is context-specific, each institution must identify feasible components in order to come up with the prioritized opportunity for improvement of EGDT adherence.

Conclusions
Adherence determination based on selected clinical importance alone might result in an incorrectly estimated benefit of clinical guideline, especially in a resource-limited setting. This pitfall of measurement may confound the effect of EGDT sepsis guideline on clinical outcomes. Both clinical importance and feasibility should be integrated into the development of adherence assessment criteria.
Supporting information S1