Introduction

Severe infection causing organ failure—sepsis—is a common problem among emergency department (ED) patients associated with substantial costs, morbidity, and mortality [1,2]. Prompt antibiotics are associated with improved sepsis outcomes [3–5], and are key features of international guidelines and regulatory mandates [6,7].

Numerous non-clinical factors influence the timing of sepsis care [8–12]. Reported approaches to overcome these barriers and improve ED sepsis care vary with respect to the type, number, and intensity of interventions. Team-based strategies meant to encourage parallel (rather than sequential) task performance and help clinicians develop a shared mental model for patients’ care are common components of these interventions. More complex and intensive interventions, however, may entail substantial costs [13] and increase the likelihood of harmful unintended effects.

We sought to investigate whether lower-intensity interventions can meaningfully improve sepsis care delivery. To accomplish this goal, we performed a retrospective, quasi-experimental analysis of sepsis door-to-antibiotic times based on a natural experiment in which ED care for all mid- and low acuity ED patients was reorganized around early, routine multidisciplinary evaluation and care planning—an approach to care sometimes termed “swarming” [14,15]— for all ED patients.

Methods

Intervention

During the pre-intervention/baseline period, patients assigned a triage acuity score of 2–5 during initial triage thereafter underwent sequential assessment by a critical care technician, nurse, and physician or advanced practice clinician. Tasks performed included helping the patient change into a patient gown and settle into the ED bed (technician and/or nurse), gathering vital signs and connection to telemetry monitor (technician and/or nurse), history taking (physician and nurse), physical examination (physician and nurse), bedside clinical documentation (nurse), IV placement (technician), and phlebotomy (technician). Care delivery in the two control EDs remained unchanged during the post-intervention period.

As part of an effort to improve the efficiency of care, beginning May 1, 2016, the intervention ED reorganized initial evaluation of all patients using a simplified version of the “swarming” approach (Table 1) already applied to patients with serious trauma, ST-elevation myocardial infarction, cardiopulmonary instability, and other patients assigned the highest triage acuity score [14,15]. Under this paradigm, ED nurses signaled readiness for “swarming” of a patient by setting to green, yellow, or red an indicator light (Luxafor, Riga, Latvia) attached to their computer screen in the nursing work station. ED physicians activated the swarm by asking a “green status” nurse and a critical care technician to join them for a patient swarm. This core ED clinical team then proceeded to perform simultaneously their initial patient evaluation and clinical tasks as described above. In most circumstances, the swarm occurred concurrent with patients being “settled” into the ED room by the nurse and critical care technician but, when a physician was unavailable at the time of a patient’s initial room placement, “settling in” tasks and the remainder of swarming could occur sequentially. The physician and nurse jointly gathered a history, thereby avoiding repeating each other’s questions and allowing each clinician to learn from and build upon history obtained by the other. Initial diagnostic testing and clinical care (e.g. collection of specific laboratory tests, initiation of intravenous fluids) was begun immediately based on verbal communication of patient care orders entered concurrently in the patient’s chart. The team then discussed the initial clinical impression and plan in the patient’s presence prior to leaving the room, helping patients understand their care plan and facilitating a shared mental model amongst the clinical team regarding the diagnostic and treatment plan and the priority of each of its components. Patients were also given an opportunity to clarify their history and ask questions. Care delivery processes occurring subsequent to the initial assessment were unchanged.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Potential mechanisms of effect of “swarming” on sepsis door-to-antibiotic time.

https://doi.org/10.1371/journal.pone.0232794.t001

While swarming was adopted as the default care model, it was not formally mandated for all patients; we were not able to identify which specific post-implementation patients actually received their care under this paradigm. From mid-May through late September 2016, ED nurses were asked to semi-formally record each swarming event by affixing an indicator sticker to a swarming documentation worksheet. Swarming adherence was calculated daily as the number of swarming events divided by the intervention ED’s patient census. Because patient-level data were not stored and some swarming events were not recorded, swarming adherence was measured on the entire ED population rather than the specific population targeted for the present analysis and may underestimate actual adherence.

Analysis

The primary exposure was ED admission after May 1, 2016 at the intervention ED. The primary outcome was door-to-antibiotic time, defined as the time from ED registration to administration of the first eligible IV- or IV-equivalent antibiotic. Secondary outcomes were antibiotic administration within 3 hours of ED arrival, ED length of stay, time from ED registration to initial physician evaluation, and inpatient mortality. Continuous data are reported as median values with interquartile range and compared using Mann-Whitney rank-sum or Kruskal-Wallis tests as appropriate. Categorical data were compared using Chi-square tests.

The primary, prespecified analysis employed a well-established quasi-experimental method called difference-in-differences analysis. Specifically, we used multivariable linear or logistic regression with robust standard error estimation to compare post-intervention and pre-intervention outcomes adjusted for a prespecified set of confounders potentially associated with time of year, study ED presentation, and door-to-antibiotic time: age, sex, Charlson Comorbidity Index, initial GCS ≤14, initial systolic blood pressure, ED triage acuity score, and arrival to the ED via ambulance. Inclusion in the multivariable model of dichotomous indicator variables for ED arrival period (pre-intervention versus post-intervention), study site (intervention versus control), and an interaction term between arrival period and site provides an estimate of the effect of the intervention with the studied outcomes that excludes temporal changes—“secular trends”— in the measured outcome unrelated to the study intervention (Fig 1). To assess the impact of not accounting for secular trends when estimating the effect of the intervention, we repeated the adjusted analyses as a simple pre/post analysis using data only from the intervention ED. Since instituting an early, parallel, multidisciplinary approach to patient assessment usually reserved for the very highest acuity patients might be expected to have the greatest effect in lower acuity patients—i.e. patients whose evaluation clinical teams might otherwise assign a lower priority—we conducted exploratory sensitivity analyses that repeated the difference-in-differences analysis of the primary outcome among subjects with (1) low triage acuity scores (3–5 on the 5-point CTAS scale) and (2) initial qSOFA scores of 0. Finally, we also repeated the primary analysis among (1) patients who had a blood culture collected while in the ED and (2) patients with an explicit discharge diagnosis for severe sepsis or septic shock based on International Classification of Disease Clinical Modification version 9 (995.92 or 785.52) or version 10 (R65.20 or R65.21) discharge diagnosis codes.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Schematic diagram of difference-in-differences analysis.

Solid lines represent hypothetical observed data for each emergency department (ED); dotted lines represent the data expected without intervention. The point estimates of analysis period means are indicated by solid circles.

https://doi.org/10.1371/journal.pone.0232794.g001

Analyses were performed using Stata version 16.0 (StataCorp LLC, College Station, TX). A two-tailed p value ≤0.05 was considered statistically significant. A priori power estimates assuming 400 eligible intervention ED patients and 700 eligible control ED patients and a 15-minute decrease in pre-intervention versus post-intervention door-to-antibiotic time at the control EDs estimated 80% power for our difference-in-differences analysis to detect a 26-minute post-intervention change in door-to-antibiotic time at the intervention site.

Results

Among 11,094 ED encounters at the intervention hospital from mid-May through late September, 6,277 (57%) had a swarm activated, 6,277 (57%) of all ED patients had a swarm event activated. Median daily adherence to the swarming protocol was 58% (interquartile range 50–66%). A total of 3,250 adult patients presented to a study ED during the pre- or post-implementation period who had a SOFA score ≥2 points above baseline and received an eligible antibiotic in the ED (Fig 2). Twenty patients assigned the highest triage score were excluded from analysis. Data for calculation of baseline SOFA was available for 32% of subjects; a baseline SOFA score of 0 was assigned to other subjects. In the random sample of 100 included subjects, 91% had confirmed infection on final adjudication.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. CONSORT-style diagram.

https://doi.org/10.1371/journal.pone.0232794.g002

Included subjects presenting to the intervention ED before versus after implementation of the “swarming” intervention were similar, as were subjects presenting in the pre- and post-implementation periods at control EDs (Table 2). Compared to the intervention ED, control ED patients were older, more likely to be female, and less likely to be of Hispanic ethnicity or non-white race.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Subject demographic and clinical characteristics.

https://doi.org/10.1371/journal.pone.0232794.t002

Median door-to-antibiotic time at the intervention ED was 157 (IQR 107–216) minutes during the pre-implementation baseline period and 151 (IQR 102–205) minutes during the post-implementation period (p = 0.11, Table 3). Inpatient mortality was low both before (2.1%) and after (1.4%) the intervention. Other unadjusted outcomes are shown in Table 3.

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. Unadjusted study outcomes by study site and intervention period.

https://doi.org/10.1371/journal.pone.0232794.t003

In the primary analysis employing multivariable adjustment for patient characteristics within a difference-in-differences framework to account for secular trends in outcomes unrelated to the study intervention, there was no difference in door-to-antibiotic times after care reorganization at the intervention ED (-7 minutes, 95% CI -20 to +6 minutes, p = 0.29). Other secondary outcomes were also unchanged after implementation of swarming (Table 4). Swarming implementation was also not associated with changes in door-to-antibiotic time in sensitivity analysis focused on patients with low triage scores and qSOFA score of 0 (Table 5). Results were also similar when analyses were restricted to patients who had a blood culture obtained in the ED or had an explicit discharge diagnosis of severe sepsis or septic shock.

Download:

• PNG
  larger image
• TIFF
  original image

Table 4. Estimated effect of ED “swarming” based on adjusted difference-in-differences analysis and simple multivariable regression.

https://doi.org/10.1371/journal.pone.0232794.t004

Download:

• PNG
  larger image
• TIFF
  original image

Table 5. Sensitivity analyses of ED “swarming’s” estimated effect on door-to-antibiotic time.

https://doi.org/10.1371/journal.pone.0232794.t005

Compared to difference-in-differences analyses, simple pre/post multivariable analyses focused only on the intervention ED yielded larger effect estimates, most of which were statistically significant (Table 4). In particular, door-to-antibiotic time was estimated to decrease by 14 minutes (95% CI 3–24 minutes, p = 0.009).

Discussion

In this study, we used ED care reorganization for mid- and low-acuity patients as a natural experiment to estimate the stand-alone effect of “swarming” on sepsis care processes and outcomes. We found no significant change in door-to-antibiotic times based on a rigorous quasi-experimental design that used contemporaneous controls to account for changes in the outcome unrelated to the study intervention. Analyses that adjusted only for patient characteristics and used a simple pre/post design, by contrast, would have suggested that the care redesign was associated with significantly shorter door-to-treatment times.

Unlike stroke and myocardial infarction—serious conditions presenting to the ED for which outcomes are similarly dependent on treatment timing—sepsis has a much more subtle clinical presentation and no simple, standardized test to quickly confirm or exclude the diagnosis. Sepsis case identification instead requires clinician judgment synthesizing patients’ symptoms, history and risk factors, physical examination, laboratory test results, and imaging findings. Driven partly by regulatory mandates and public reporting requirements [7,24], many hospitals are currently investing substantial resources in efforts to improve sepsis care delivery.

Swarming has been proposed as a method to improve the efficiency of ED care by reducing duplicative serial evaluations and helping the treating team develop a shared mental model regarding the patient’s evaluation and treatment [14,15]. We wondered whether this relatively simple, low-intensity, and broadly-targeted intervention intended to expedite evaluation for all mid- and low-acuity ED patients would have a moderate but nonetheless meaningful impact on sepsis patients’ door-to-antibiotic time. While we cannot rule out the possibility that alternative low-intensity interventions or 100% adherence to swarming would have shown benefit, our findings suggest that simply bringing the clinical team together at the bedside may be insufficient to improve ED-based care delivery for patients with sepsis. Instead, more intensive, focused, and multimodal interventions may be required to meaningfully improve sepsis care. Besides broadening membership of the swarming team (in particular, by adding a pharmacist), a more effective sepsis-specific protocol might integrate automated tools for very early identification of patients at risk of sepsis, standardized protocols to facilitate rapid completion of diagnostic tests, ongoing evaluation of barriers to sepsis protocol adoption and effectiveness, and/or individual case review and clinician feedback. However, stand-alone team-based interventions like swarming could also have beneficial effects on other outcomes—including patients’ care experience, clinical team members’ satisfaction, and medical errors— which we were unable to measure in the present analysis. While prior data suggest care processes targeted by swarming are important determinants of antibiotic timing [10,11,25–27], the absence of statistically significant association could reflect swarming’s failure to influence other rate-limiting processing preceding antibiotic initiation (e.g. antibiotic order review and fulfilment by pharmacy).

Strengths of this study include an analytic approach more conducive to causal inference than simple pre/post analysis and an approach to confounder adjustment consistent with recent guidelines [28,29]. In fact, use of simple pre/post design for this analysis would have suggested the swarming intervention was associated with a moderate but statistically significant decrease in door-to-antibiotic time. This discrepancy between the results of simple pre/post analysis and our quasi-experimental analysis highlights the methodological problems with studies of system-level interventions that fail to account for secular trends.

Our study has several limitations. The studied EDs shared a common sepsis protocol throughout the study, but unrecognized interventions affecting sepsis care that were unique to the control EDs could have led to under-estimation of swarming’s effect at the intervention hospital. Other potential issues derive from our guideline-adherent approach to sepsis case identification. About 1 in 10 study subjects did not prove to have infection on final review, a value similar to the true-positive infection rate (88%) reported by Rhee et al. in a study where the overall positive-predictive value of electronic sepsis case identification was 70% [2]. However, analyzing patients with suspected infection at the time of ED evaluation is consistent with sepsis consensus definitions and, more importantly given the questions investigated here, the clinical decisions confronting ED clinicians. To wit, the final determination of sepsis status is usually unavailable at the time ED clinicians must make therapeutic decisions. Our method for identifying patients for study enrollment may also have misclassified some patients with simple infection due to either (1) a lack of causal association with concurrent acute organ failure or (2) overestimation of acute organ failure severity resulting from missing data for calculation of baseline SOFA scores. Finally, we excluded sepsis patients whose infection was unrecognized or untreated during the ED stays, as well as patients whose organ failure was not detectable or not present before ED discharge. We were unable to assess whether swarming altered the likelihood of sepsis diagnosis, infection treatment, or organ failure detection during the ED encounter.

The most important limitation of this study, however, was our inability to identify whether specific ED patients actually received swarm-based care. Uptake was fairly high during the initial months after implementation but anecdotally may have waned toward the end of 2016. For this reason, we a priori closed the data analysis period in mid-November, 2016. While our study should accurately reflect the impact associated with a pragmatic implementation of ED swarming, we cannot exclude the possibility that results would have differed if the analysis had been restricted to patients who received a swarm-based evaluation or if additional resources had been dedicated to promoting and maintaining adherence to the swarming model. Appropriately designed and controlled prospective hybrid implementation/effectiveness trials could not only quantify whether interventions like swarming improve sepsis care but would also provide empiric data on why such interventions succeed or fail.

Conclusion

Introduction of “swarm”-based evaluation for mid- and lower-acuity ED patients was not associated with improvements in door-to-antibiotic time for patients with sepsis after accounting for secular trends in sepsis care processes and outcomes.