Skip to main content
Advertisement
  • Loading metrics

Use of a continuous single lead electrocardiogram analytic to predict patient deterioration requiring rapid response team activation

  • Sooin Lee,

    Roles Conceptualization, Data curation, Formal analysis, Methodology, Validation, Visualization, Writing – original draft, Writing – review & editing

    Affiliation Fifth Eye, Inc, Ann Arbor, Michigan, United States of America

  • Bryce Benson,

    Roles Data curation, Investigation, Visualization, Writing – original draft, Writing – review & editing

    Affiliation Fifth Eye, Inc, Ann Arbor, Michigan, United States of America

  • Ashwin Belle,

    Roles Conceptualization, Investigation, Methodology, Project administration, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

    Affiliation Fifth Eye, Inc, Ann Arbor, Michigan, United States of America

  • Richard P. Medlin,

    Roles Data curation, Investigation, Validation, Writing – review & editing

    Affiliations Department of Emergency Medicine, University of Michigan, Ann Arbor, Michigan, United States of America, Max Harry Weil Institute for Critical Care Research and Innovation, University of Michigan, Ann Arbor, Michigan, United States of America

  • David Jerkins,

    Roles Investigation, Validation, Writing – review & editing

    Affiliations Max Harry Weil Institute for Critical Care Research and Innovation, University of Michigan, Ann Arbor, Michigan, United States of America, Surgical Intensive Care Unit and Rapid Response Team, University of Michigan, Ann Arbor, Michigan, United States of America

  • Foster Goss,

    Roles Methodology, Writing – review & editing

    Affiliation Department of Emergency Medicine, University of Colorado—Anschutz, Aurora, Colorado, United States of America

  • Ashish K. Khanna,

    Roles Methodology, Writing – review & editing

    Affiliation Department of Anesthesiology, Section on Critical Care Medicine, Perioperative Outcomes and Informatics Collaborative (POIC), Wake Forest School of Medicine, Atrium Health Wake Forest Baptist, Winston-Salem, North Carolina, United States of America

  • Michael A. DeVita,

    Roles Conceptualization, Methodology, Writing – review & editing

    Affiliation Department of Clinical Medicine, Columbia Vagelos College of Physicians and Surgeons, New York, New York, United States of America

  • Kevin R. Ward

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

    keward@umich.edu

    Affiliations Department of Emergency Medicine, University of Michigan, Ann Arbor, Michigan, United States of America, Max Harry Weil Institute for Critical Care Research and Innovation, University of Michigan, Ann Arbor, Michigan, United States of America

Abstract

Identifying the onset of patient deterioration is challenging despite the potential to respond to patients earlier with better vital sign monitoring and rapid response team (RRT) activation. In this study an ECG based software as a medical device, the Analytic for Hemodynamic Instability Predictive Index (AHI-PI), was compared to the vital signs of heart rate, blood pressure, and respiratory rate, evaluating how early it indicated risk before an RRT activation. A higher proportion of the events had risk indication by AHI-PI (92.71%) than by vital signs (41.67%). AHI-PI indicated risk early, with an average of over a day before RRT events. In events whose risks were indicated by both AHI-PI and vital signs, AHI-PI demonstrated earlier recognition of deterioration compared to vital signs. A case-control study showed that situations requiring RRTs were more likely to have AHI-PI risk indication than those that did not. The study derived several insights in support of AHI-PI’s efficacy as a clinical decision support system. The findings demonstrated AHI-PI’s potential to serve as a reliable predictor of future RRT events. It could potentially help clinicians recognize early clinical deterioration and respond to those unnoticed by vital signs, thereby helping clinicians improve clinical outcomes.

Author summary

Recognizing patient deterioration remains challenging even for experienced clinicians and nurses. RRTs can help mobilize resources to respond to patients earlier. However, determining when to activate RRTs is difficult. We retrospectively evaluated a software as a medical device, AHI-PI, compared the vital signs of heart rate, blood pressure, and respiratory rate to understand if AHI-PI could provide an earlier indicator of patient deterioration than vital signs. Our findings demonstrated AHI-PI’s potential to serve as a reliable predictor of future RRT events, before vital sign changes occur. This could potentially help clinicians recognize patients at risk for clinical deterioration and improve clinical outcomes through early targeted therapy or interventions.

Introduction

Recognizing signs of early physiologic instability in hospitalized patients is critical to improving outcomes and preventing mortality. Failing to do so can contribute to unexpected transfers to a higher level of care, increased lengths of stay, and even unexpected deaths. Studies show that deliberate and consistent monitoring of vital signs improves early detection and clinical action [1,2]. However, measuring and documenting vital signs remain inconsistent and error prone in practice [3,4]. As a result, there may be delays in both recognizing and treating a patient at risk for deterioration. There are additional compounding factors that increasingly contribute to this problem. A few examples of such factors are nursing shortages, growing clinical workloads, patient comorbidities, data limitations, and resource constraints [5].

As a mitigation strategy, hospitals in many countries and across the United States have developed rapid response systems designed to detect deterioration early and rapidly respond to those patients believed to be deteriorating. These systems have teams that are called by different names and include rapid response teams (RRTs), medical emergency response teams, high acuity response teams, code teams, etc. [6]. These response teams are typically multidisciplinary including physicians, nurses, respiratory therapists, and pharmacists supporting deteriorating patients to prevent potentially avoidable adverse events such as cardiopulmonary arrest [7]. However, even with widespread adoption of RRTs, their effectiveness continues to be debated and evaluated [6,810].

The detection and activation of rapid response is prone to delays. A common cause of delay is caused by infrequent vital signs checks [11]. Unexpected ICU admissions burden health care resources and are associated with substantial increases in cost of care, length of hospital stay and a nearly 20% increase in mortality [12]. While there is potential for responding to patients earlier with better vital sign monitoring and RRTs, identifying the onset of patient deterioration is challenging. This naturally makes determining when to activate an RRT difficult. Clinical decision support systems that help clinicians recognize clinical deterioration prior to changes in traditional vital signs could be valuable for early intervention. Desirable characteristics of such a system would include easy automation, continuous output, simple interpretation, and no significant change or burden added to clinical workflow.

In this study, we evaluate the Analytic for Hemodynamic Instability-Predictive Index (AHI-PI: Fifth Eye, Ann Arbor, MI), a FDA approved software as a medical device (SaMD) that uses a continuous ECG waveform and extracted heart rate variability (HRV) to predict hemodynamic instability [1315]. Changes in HRV have been demonstrated to reflect changes in the autonomic nervous system in the setting of many states of critical illness and injury including hemorrhage, sepsis, cardiogenic shock, respiratory failure, and others with these changes occurring prior to overt decompensation [1622]. In previous studies, we demonstrated its ability to predict hemodynamic instability (a combination of tachycardia and hypotension) with high sensitivity and specificity with average lead times greater than 3 hours [15,23].

AHI-PI automates the extraction and analysis of ECG patterns to reflect the compensatory burden on the autonomic nervous system to provide information regarding the patient’s predicted future risk for clinical deterioration based on the known physiologic relationship of HRV, the autonomic nervous system, and the cardiovascular system [16,19]. It also includes a signal quality assessment and processing of extracted patterns through a pretrained classification model that embeds nonlinear HRV complexity and ECG morphologic features into a single output [1315]. AHI-PI builds on this output and updates every two minutes, producing one of three types of outputs, red, yellow, or green, indicating high, moderate, or low risk respectively of a future episode of hemodynamic instability [23]. Hemodynamic instability is defined as a combination of heart rate greater than or equal to 100 beats per minute and a systolic blood pressure (SBP) less than 90 mmHg or mean arterial pressure (MAP) of less than 70 mmHg.

We examine and assess the ability of AHI-PI in predicting the need for an RRT prior to its activation by comparing it to changes in the vital signs of heart rate and blood pressure, but also now add respiratory rate for hospitalized patients who were undergoing continuous ECG monitoring and in which an RRT was activated.

Methods

This was a retrospective single-center observational cohort study conducted at the University of Michigan, a quaternary academic health system in Ann Arbor, Michigan, between August 2019 and April 2020. The study dataset included consecutive hospitalized adult (≥ 18 years) patients who were undergoing continuous ECG monitoring on telemetry, stepdown and intensive care units and for whom an RRT was activated.

The study was approved by the University of Michigan Institutional Review Board (HUM00092309). Due to its retrospective design and use of deidentified data, a waiver of consent was granted.

Patient data including demographic and vital signs were extracted from Epic Systems (Verona, WI) electronic health record (EHR). ECG waveform data were obtained from an external data warehouse (EDW) that collects, stores, and maintains a high-resolution physiologic signal database of patients including real-time physiologic signals and waveforms such as ECG. The ECG was sampled at 240 Hz for this study.

RRT event

The study defined an RRT event as a rapid response event including cardiac arrest (code calls) as recorded in the EHR. AHI-PI outputs and EHR-recorded vital signs within 48 hours before each RRT event were analyzed in this study to account for out-of-range vital signs that were documented before activation of the RRT [2426]. To ensure that no redundant AHI-PI or vital signs data were used for patients having more than one RRT event, only those at least 48 hours apart were chosen, starting with the first one in chronological order.

Study dataset

The study data initially contained a total of 168 RRT events which were at least 48 hours apart. Patients with any exclusions (i.e., heart transplants, ventricular assist device, and chronic arrhythmias such as atrial fibrillation) were removed from the analysis as these conditions makes HRV an unreliable predictor [27,28]. Patients without any valid AHI-PI outputs in the previous 48 hours prior to the event were also excluded from the analysis leaving 110 RRT events for analysis. Blinded to AHI-PI’s results, two physicians from Michigan Medicine then reviewed each event to determine if each of the RRT calls resulted in a true escalation of care (e.g., need for transfer to the ICU, institution of vasopressors, mechanical or noninvasive ventilation for respiratory failure, blood transfusion, or cardiopulmonary resuscitation at the time as recorded in the EHR). After clinician review, there were 96 RRT events from 91 patients identified (Fig 1).

AHI-PI and vital signs risk indication

The study defined the presence of risk indication as having AHI-PI high or moderate risk output within 48 hours before a given RRT event. That is, if AHI-PI had one or more high or moderate risk outputs within 48 hours before an event, then AHI-PI was said to have indicated risk before the event. For vital signs, the presence of risk indication was conservatively defined as having hypotension (SBP < 90mmHg or MAP < 70mmHg) combined with either tachycardia or tachypnea (pulse ≥ 100 beats per minute or respiratory rate > 25 breaths per minute). That is, if hypotension and tachycardia or hypotension and tachypnea were recorded within 48 hours before the event, then vital signs were said to have indicated risk. The combination of tachycardia or tachypnea combined with hypotension, in our opinion, gives high confidence that the “out-of-range” period represents a conservative ground truth indicator of the presence of hemodynamic instability [29,30].

AHI-PI and vital signs initial risk indication time

In addition to looking at the presence of risk indication in AHI-PI and vital signs in the previous 48 hours of an RRT event, the study calculated how long before the event AHI-PI and vital signs first indicated risk within the analysis timeframe (Fig 2). For AHI-PI, the initial risk indication time was measured as the distance from the RRT event time to the first time AHI-PI produced high or moderate risk output within 48 hours before the event. For vital signs, the initial risk indication time was measured as the distance from the event time to the first time both hypotension and tachycardia or tachypnea were present within the previous 48 hours of the event. For example, if tachycardia was recorded 10 minutes after presence of hypotension in the EHR, then the time when tachycardia was recorded would be used to calculate vital signs’ initial risk indication time.

thumbnail
Fig 2. AHI-PI and vital signs initial risk indication time.

This example shows hypotension and tachycardia having occurred prior to RRT activation. Icons courtesy of Fifth Eye, used under a CC-BY license.

https://doi.org/10.1371/journal.pdig.0000465.g002

Case-control study

Even though the main goal of the study was to evaluate AHI-PI in situations that required RRTs specifically, the study also sought to evaluate AHI-PI’s performance in patients where RRTs were not called by examining a case-control cohort. For each of the patient-encounters included in this RRT study, a cohort without any RRT event was identified and paired with it for comparison. Each control case was matched 1:1 to each RRT cases by age (+/- <6 years), sex, unit type, primary diagnosis, and ECG monitoring duration. Matching for the ECG monitoring duration allowed assigning each control case an event time such that the event happened proportionately at the same time as its matched RRT case. A more detailed description of the steps taken to identify control cases is in S1 Appendix.

A two-sample t-test and conditional logistic regression were used to compare percentages of AHI-PI risk indication within 48 hours before the events in the control and RRT cases. In all cases, AHI-PI was running and executed silently in a background environment not accessible to clinicians or care teams; therefore, no interventions to mitigate risk identified by AHI-PI were taken at any stage.

Results

The mean patient age was 61.25 ± 14.36 years. Patient demographics are provided in Table 1. AHI-PI indicated risk in 92.71% (89) of the 96 RRT events (Fig 3). All 89 events (subgroup 1) had AHI-PI high or moderate risk outputs within 48 hours before the RRT. In contrast, EHR documented vital signs indicated risk in 41.67% (40) of the 96 RRT events, with all 40 events having documented hypotension and tachycardia or tachypnea in the previous 48 hours. In these 40 events, both vital signs and AHI-PI indicated risk (subgroup 1A). Where documented vital signs indicated no risk before the RRT event, AHI-PI indicated risk in 51.04% (49) of events (subgroup 1B). The study found 2.08% (2) of the 96 RRT events showed risk indication with vital signs’ and without AHI-PI’s (subgroup 2A) and 5.21% (5) of the 96 events had neither vital signs nor AHI-PI presented risk indication prior to the events (subgroup 2B). Results are summarized in the contingency table in Fig 3 and reasons for RRT calls are shown in Table 2.

thumbnail
Fig 3. Table and flowchart summarizing presence (Y) / absence (N) of risk indication.

https://doi.org/10.1371/journal.pdig.0000465.g003

The exact analysis timeframes varied across the 96 RRT events because they depended on the durations of ECG monitoring preceding the events. The median analysis timeframe, or ECG monitoring duration, across the events was 45.88 hours, with a mean of 30.38 hours. Overall, AHI-PI outputs were available and valid the majority of time, with a median percentage of available outputs being 89.29%. That is, AHI-PI produced few unavailable, or invalid, outputs across the events during the analysis timeframes. The distributions of ECG monitoring durations as well as of the counts and percentages of available AHI-PI outputs for the 96 events are shown in Table 3.

thumbnail
Table 3. Numerical summaries of ECG monitoring durations and of the counts and percentages of available AHI-PI outputs.

https://doi.org/10.1371/journal.pdig.0000465.t003

AHI-PI initial risk indication time by subgroup

A summary of AHI-PI initial risk indication times for subgroups 1, 1A, and 1B can be found in Fig 4. Across all 89 events (subgroup 1), AHI-PI’s initial risk indication time was 26.48 hours before the events, on average (median: 28.57 hours). The mean initial risk indication times were 30.54 and 23.17 hours for subgroups 1A and 1B, respectively (medians: 42.67 and 21.80 hours). Comparing subgroup 1A and subgroup 1B using Mann-Whitney U test, the study did not find sufficient evidence to conclude that the initial risk indication time for AHI-PI in subgroup 1A is different from that in subgroup 1B (p-value: 0.1113).

thumbnail
Fig 4. Distributions of AHI-PI initial risk indication times for the three subgroups.

https://doi.org/10.1371/journal.pdig.0000465.g004

AHI-PI risk indication in and outside of ICU

Of the 96 RRT events, 60 (62.5%) occurred within ICUs and 36 (37.5%) outside of ICUs. The study examined the 60 and 36 events, separately, and found their results to be consistent with the results from all care units regarding how AHI-PI and vital signs indicated risk. A higher proportion of events had risk indication by AHI-PI than by vital signs in and outside of ICUs. Unit specific distributions of AHI-PI’s initial risk indication times are shown in Fig 5 and exhibit similar distributions as in Fig 4. Across all of subgroup 1, the mean AHI-PI initial risk indication times were 28.00 and 24.25 hours before the RRT events for the events in and outside of ICUs, respectively. For subgroup 1A, the mean initial risk indication times were 30.05 and 31.68 hours in and outside of ICUs. The mean times were 25.71 and 20.53 hours in and outside of ICUs for subgroup 1B, respectively.

thumbnail
Fig 5. Distributions of AHI-PI initial risk indication times for the three subgroups by unit type.

https://doi.org/10.1371/journal.pdig.0000465.g005

Comparison between AHI-PI and vital signs for subgroup 1A

As previously described and shown in Fig 3, there were 40 RRT events where both AHI-PI and vital signs indicated risk (subgroup 1A). These 40 events’ initial risk indication times distributions for AHI-PI and vital signs are shown in Fig 6. AHI-PI’s initial risk indication time was 30.54 hours, on average (median: 42.67 hours, range: [0.26 hour, 48.0 hours], interquartile range: [4.66 hours, 47.95 hours]). In contrast, the mean vital signs initial risk indication times was 19.26 hours before the events, respectively (median: 10.87 hours, range: [0.09 hour, 47.92 hours], interquartile range: [1.51 hours, 40.18 hours]). The study found sufficient evidence that AHI-PI’s initial risk indication time was earlier than vital signs’ (p-value: 0.0062 (Two-sample t-test)) and that their distributions are not equal (p-value: 0.0022 (Mann-Whitney U test)). Additionally, in 32 of the 40 events, AHI-PI indicated risk before it was evident in vital signs. Comparing these 32 events’ initial risk indication times, AHI-PI first indicated risk 15.87 hours before vital signs, on average (median: 8.20 hours).

thumbnail
Fig 6.

Boxplots showing the distribution of vital signs initial risk indication times (top) as well as the distribution of AHI-PI initial risk indication times (bottom) for subgroup 1A.

https://doi.org/10.1371/journal.pdig.0000465.g006

The study also found that, once AHI-PI indicated risk for the first time, its outputs often showed high or moderate risk leading up to the RRT events. As shown on the far right of Fig 7, the mean and median percentage of risk outputs between the initial risk indication time and event time were 67.57% and 76.36%, respectively (interquartile range: [40.21%, 100.0%]). The study also looked at the distributions of SBP, MAP, respiratory rate, and heart rate values after vital signs’ initial risk indication time, leading up to the event time (Fig 7). The mean and median SBP were 108 mmHg and 105 mmHg, respectively. The mean MAP was 77 mmHg (median: 75 mmHg). The mean and median respiratory rates were 24 and 22 breaths per minute. The mean pulse was 101 bpm, with a median of 101 bpm. The study additionally looked at AHI-PI outputs within 30 minutes of the events to evaluate if this pattern of strong risk indications remained consistent near the event times across all the 37 events. The mean percentage of risk outputs was 77.30%, with a median of 100.0% (interquartile range: [93.33%, 100.0%]). This showed that the risk indication strength remained strong in the last half hour prior to the events.

thumbnail
Fig 7.

Distributions of EHR-recorded SBP, MAP, respiratory rate, and pulse values between the vital signs initial risk indication time and event time and of percentages of AHI-PI risk indications between AHI-PI initial risk indication time and event time (subgroup 1A). Y-axis label ‘bpm*’ is either beats per minute (heart rate) or breaths per minute (respiratory rate) depending on the context.

https://doi.org/10.1371/journal.pdig.0000465.g007

In addition to finding that AHI-PI indicated risk earlier than vital signs when all units are combined (Fig 6), the study also found this within ICUs and outside of ICUs. This consistent finding is depicted in Fig 8. In fact, for subgroup 1A where both AHI-PI and vital sign changes occurred before RRT, AHI-PI was earlier than vital signs in 23 of the 28 events within ICUs by an average of 13.77 hours and in 9 of the 12 events outside of ICUs by an average of 21.25 hours.

thumbnail
Fig 8.

Distributions of initial risk indication times for vital signs and AHI-PI by unit type (subgroup 1A).

https://doi.org/10.1371/journal.pdig.0000465.g008

Case-control study

Lastly, the study compared the percentages of AHI-PI risk indication within 48 hours before the events in the RRT group to those in the control group by matching each control case 1:1 to each RRT case by age, sex, unit, diagnosis, and ECG monitoring duration. The study found that the matched control group of 96 cases had substantially fewer AHI-PI risk outputs, compared to the RRT group, overall. As shown in Table 4, the median percentage of risk indication was 12.60% for the control group, meaning that at least half of the controls had 12.60% or less of their AHI-PI outputs indicating risk. On average, 33.12% of the outputs indicated risk across all the control cases. On the other hand, the median percentage of risk indication outputs was 48.85% for the RRT group, with a mean of 49.56%. The average percentage of AHI-PI risk indication for the control group is noticeably shifted upwards, compared to the RRT group whose average is close to its median. This is because matching each control case 1:1 to each of the RRT cases by characteristics such as unit type and primary diagnosis methodically led the control group to contain patients who were at similar risk for RRT as the actual RRT group. That is, while some of those in the control group did experience hemodynamic deterioration, none required RRTs. By design, what mainly distinguishes the control group from the RRT group is that the controls did not have RRT events. In fact, 65 cases in the control group had at least one AHI-PI risk indication at some point before the assigned event times. Of the 65, 15 (23.08%) showed out-of-range vital signs before the event times, and 53 (81.54%) received fluids as a bolus or pressors during their encounters (but no RRT activation). A total of 9 out of the 65 did not receive such interventions and did not have out of range vital signs. Even so, the study found sufficient evidence that the percentage of AHI-PI risk indication was lower in the control group, compared to the RRT group, on average (p-value: 0.0019 (Two-sample t-test)).

thumbnail
Table 4. Numerical summaries of the percentages of AHI-PI risk indication for RRTs and for controls.

https://doi.org/10.1371/journal.pdig.0000465.t004

The conditional logistic regression results are summarized in Table 5. The study found that the events requiring RRTs were more likely to have AHI-PI risk indication, compared to the events where RRTs were not needed or avoided. Specifically, compared to those without any AHI-PI risk indication, those with AHI-PI risk indication had 9 times the odds of having RRT events (odds ratio: 9.0).

Discussion

Prevention of morbidity and mortality using RRTs can only work if high risk deteriorating patients are rapidly and reliably identified, and RRT calls are triggered. Few studies have worked on preventing physiologic decline before achieving the triggering criteria. While evidence exists that changes in vital signs can be helpful in the early identification of patients who may benefit from an RRT, identifying the early onset of clinical deterioration by vital sign measurements can be challenging due to various factors such as infrequent, inaccurate, or delayed vital signs documentation and suboptimal nursing to patient ratios [5]. This can make it difficult to determine when to activate an RRT. As such there continues to be a need to develop clinical decision support tools that could optimize RRT utilization and efficacy by recognizing the potential for clinical deterioration at earlier time points. While largely theoretical, earlier identification of patients in need would allow for assessment and interventions that could potentially reduce the need for transfer to the ICU or increase earlier transfer to the ICU resulting in reduced total ICU stay and prevent catastrophic events such as cardiac arrest.

Before any prospective use of AHI-PI as a clinical decision support tool for RRT activation, the study sought to evaluate AHI-PI’s potential to contribute as such a support system using a retrospective analysis. The study examined patients in whom a RRT activation occurred (including code events for cardiac arrest), among all consecutive non-contraindicated adult patients with available ECG monitoring data. The study analyzed and compared AHI-PI to vital signs for the potential to serve as an earlier indicator and derived insights from these RRT events that have important implications in support of AHI-PI’s efficacy as a clinical decision support system.

Of the 96 RRT events included in the study, a significantly higher proportion of the events had risk indication by AHI-PI (92.71%; subgroup 1) than by vital signs (38.54%). All the events for which vital signs indicated risk in advance were also predicted by AHI-PI (subgroup 1A). Only 2 of all 96 RRTs (2.08%; subgroup 2A) were preceded by abnormal vital signs and not AHI-PI. Additionally, AHI-PI’s initial risk indication time was more than a day in advance of the events, on average. These findings support AHI-PI’s potential to provide additional clinical insights hours in advance of the need for RRT, which may help clinicians intervene ahead of time to avoid deterioration. Furthermore, all these findings were consistent among both events within ICUs and events outside of ICUs. The consistency across units supports AHI-PI’s potential as a clinical decision support system.

The study also found that AHI-PI indicated risk for the majority of the events without vital signs’ risk indication (subgroup 1B). This finding supports that AHI-PI could be used to help respond to patients who are unnoticed by vital signs. When looking specifically at the events with risk indication by both AHI-PI and vital signs (subgroup 1A), the study found that AHI-PI’s initial risk indication times were often earlier and more consistent in risk indication, compared to vital signs, after their respective initial indication times. These findings demonstrate AHI-PI’s potential to serve as a reliable predictor of future RRT events. Especially considering the continuous nature of AHI-PI, it could help clinicians recognize clinical deterioration sooner than when relying solely on vital signs, helping them improve clinical outcomes. Code and RRT responses from the ICUs with documentation of such events were included in this study. This is important given the increasing use of tele-ICU services and for ICUs which may be less resourced than academic medical centers [31].

The types of RRT events (Table 2) are, for the most part, indicative of physiology that would impact and produce early changes in the autonomic nervous system and heart rate variability thus making AHI-PI a potentially useful clinical decision support tool. However, events such as cardiac arrest, certain sudden dysrhythmias, stroke, pulmonary embolism, anaphylactic reactions, and others where an RRT may be called may not lend themselves to prediction and early intervention based on HRV monitoring depending on the etiology and suddenness of the event as well as recognition of symptoms by providers. AHI-PI’s performance in some of these conditions were mixed.

A case-control study was conducted to understand and evaluate how AHI-PI performed in situations without RRT calls, compared to situations with them. In general, the control group had substantially less AHI-PI risk indicators, compared to the RRT group. This further supports that AHI-PI may reliably work as part of a clinical decision support system in detecting the need for an RRT or code team prior to activation.

AHI-PI is designed to be a clinical decision support tool, providing health care providers an advanced indicator when patients may require closer evaluation or re-evaluation. While statistically significant differences exist between AHI-PI and vital signs and AHI-PI indicated an increased need for an RRT (9 times greater risk) compared to controls, there is some overlap in performance between AHI-PI and vitals. However, as a continuous output variable, AHI-PI is expected to act as a powerful complement to vital signs and especially intermittent vital signs. AHI-PI may also compliment EHR based early warning systems which in combination could increase accuracy, provide greater insight into the underlying cause of the alert, and assist in more rapid adjudication of the alarms [3235]. Furthermore, since AHI-PI is approved for use with various single lead wearable ECG patches, it may allow more intensive and informative monitoring of floor level patients who are monitored much less frequently.

Limitations

The study used retrospective data, which were not collected or designed specifically for this study. This could mean that the data contained unknown biases or confounding factors that the study cannot account for. The study also did not take into account medications that were given to prevent deterioration in the control group. Additionally, although respiratory rate has been included in this analysis, its measurement through either bedside patient assessment by care providers or by bedside monitors using traditional thoracic impedance technology has demonstrated significant disagreement with true respiratory rates [3641]. Patients with age below 18 years, heart transplants, ventricular assist device, pace maker dependency, and chronic with sustained atrial or ventricular arrhythmias were excluded forming an additional limitation to this study as these are contraindicated for the AHI-PI analytic since they are known to limit or interfere with the determination of HRV. Although the size of the case-control population in this study was sufficient to assess the potential for AHI-PI towards the application in RRT settings, a future study on this could benefit more from a larger patient population.

Lastly, because only RRT events were evaluated, the sensitivity and specificity of AHI-PI cannot be determined. As an alternative we utilized the case-control series to determine the odds ratio having an RRT event between the two populations. To what extent use of AHI-PI could have reduced RRT and code events by prompting increased vigilance and earlier interventions is unknown. However, the data presented supports the next required step which is a prospective study with a larger patient population that examines AHI-PI’s efficacy in detecting the need for an RRT prior to activation will be needed.

Conclusion

The study examined the ability of an ECG based SaMD product called AHI-PI as a clinical decision support system in detecting decompensation events requiring an RRT. The findings from this study demonstrated AHI-PI’s potential for identifying the need for RRTs early than changes in vital signs. Accordingly, AHI-PI could potentially help clinicians recognize early clinical deterioration, take necessary measures, and improve clinical outcomes. The data from this study supports future prospective trials.

Supporting information

S1 Appendix. Steps taken for conducting the case-control study.

https://doi.org/10.1371/journal.pdig.0000465.s001

(DOCX)

References

  1. 1. DeVita M. Medical emergency teams: deciphering clues to crises in hospitals. Crit Care. 2005 Aug;9(4):325–6. pmid:16137372
  2. 2. Harrison GA, Jacques TC, Kilborn G, McLaws ML. The prevalence of recordings of the signs of critical conditions and emergency responses in hospital wards—the SOCCER study. Resuscitation. 2005 May;65(2):149–57. pmid:15866394
  3. 3. Fuller T, Fox B, Lake D, Crawford K. Improving real-time vital signs documentation. Nurs Manage. 2018 Jan;49(1):28–33. pmid:29287047
  4. 4. Lockwood C, Conroy-Hiller T, Page T. Vital signs. JBI Libr Syst Rev. 2004;2(6):1–38. pmid:27820007
  5. 5. Haddad LM, Annamaraju P, Toney-Butler TJ. Nursing Shortage. StatPearls. 2023.
  6. 6. Jones DA, DeVita MA, Bellomo R. Rapid-response teams. N Engl J Med. 2011 Jul 14;365(2):139–46. pmid:21751906
  7. 7. Chan PS, Jain R, Nallmothu BK, Berg RA, Sasson C. Rapid response teams: a systematic review and meta-analysis. Arch Intern Med. 2010 Jan 11;170(1):18–26. pmid:20065195
  8. 8. Tirkkonen J, Tamminen T, Skrifvars MB. Outcome of adult patients attended by rapid response teams: a systematic review of the literature. Resuscitation. 2017 Mar;112:43–52. pmid:28087288
  9. 9. Lyons PG, Edelson DP, Churpek MM. Rapid response systems. Resuscitation. 2018 Jul;128:191–7. pmid:29777740
  10. 10. DeVita MA, Hillman K, Smith GB. Resuscitation and rapid response systems. Resuscitation. 2014 Jan;85(1):1–2. pmid:24280486
  11. 11. Chen J, Bellomo R, Flabouris A, Hillman K, Assareh H, Ou L. Delayed emergency team calls and associated hospital mortality: a multicenter study. Crit Care Med. 2015 Oct;43(10):2059–65. pmid:26181217
  12. 12. Khanna AK, Moucharite MA, Benefield PJ, Kaw R. Patient characteristics and clinical and economic outcomes associated with unplanned medical and surgical intensive care unit admissions: a retrospective analysis. Clinicoecon Outcomes Res. 2023;15:703–19. pmid:37780944
  13. 13. Belle A, Ansari S, Spadafore M, Thirumala P, Lay-Ekuakille A, Bai Y, et al. A signal processing approach for detection of hemodynamic instability before decompensation. PLoS One. 2016;11(2):e0148544. pmid:26871715
  14. 14. Belle A, Benson B, Salamango M, Islim F, Daniels R, Ward KR. A continuous real-time analytic for predicting instability in acute care rapid response team activations. Int J Med Health Res. 2020;14(11).
  15. 15. Schmitzberger FF, Hall AE, Hughes ME, Soliman H, Ahadi G, Chang K, et al. Detection of hemodynamic status using an analytic based on an electrocardiogram lead waveform. Crit Care Explor. 2022 May;4(5):e0693. pmid:35620767
  16. 16. Salahuddin N, Shafquat A, Marashly Q, Zaza KJ, Sharshir MD, Khurshid M, et al. Increases in heart rate variability signal improved outcomes in rapid response team consultations: a cohort study. Cardiol Res Pract. 2018;2018(1):1590217. pmid:29686889
  17. 17. Kleiger RE, Miller JP, Bigger JT Jr, Moss AJ. Decreased heart rate variability and its association with increased mortality after acute myocardial infarction. Am J Cardiol. 1987 Feb 1;59(4):256–62. pmid:3812275
  18. 18. Odemuyiwa O, Malik M, Farrell T, Bashir Y, Poloniecki J, Camm J. Comparison of the predictive characteristics of heart rate variability index and left ventricular ejection fraction for all-cause mortality, arrhythmic events and sudden death after acute myocardial infarction. Am J Cardiol. 1991 Aug 15;68(5):434–9. pmid:1872267
  19. 19. Bodenes L, N’Guyen QT, Le Mao R, Ferrière N, Pateau V, Lellouche F, et al. Early heart rate variability evaluation enables to predict ICU patients’ outcome. Sci Rep. 2022 Feb 15;12(1):2498. pmid:35169170
  20. 20. Stein PK, Bosner MS, Kleiger RE, Conger BM. Heart rate variability: a measure of cardiac autonomic tone. Am Heart J. 1994 May;127(5):1376–81. pmid:8172068
  21. 21. da Silva RB, Neves VR, Montarroyos UR, Silveira MS, Sobral Filho DC. Heart rate variability as a predictor of mechanical ventilation weaning outcomes. Heart Lung. 2023 May-Jun;59:33–6. pmid:36706686
  22. 22. Mollura M, Lehman LH, Mark RG, Barbieri R. A novel artificial intelligence based intensive care unit monitoring system: using physiological waveforms to identify sepsis. Philos Trans A Math Phys Eng Sci. 2021 Dec 13;379(2212):20200252. pmid:34689614
  23. 23. Benson B, Belle A, Lee S, Salamango M, Ward KR, Daniels R, et al. Prediction of episode of hemodynamic instability using an electrocardiogram based analytic: a retrospective cohort study. BMC Anesthesiol. 2023 Sep 22;23(1):324. pmid:37737164
  24. 24. Buist MD, Jarmolowski E, Burton PR, Bernard SA, Waxman BP, Anderson J. Recognising clinical instability in hospital patients before cardiac arrest or unplanned admission to intensive care: a pilot study in a tertiary-care hospital. Med J Aust. 1999 Jul 5;171(1):22–5. pmid:10451667
  25. 25. Franklin C, Mathew J. Developing strategies to prevent in-hospital cardiac arrest: analyzing responses of physicians and nurses in the hours before the event. Crit Care Med. 1994 Feb;22(2):244–7. pmid:8306682
  26. 26. Cretikos M, Chen J, Hillman K, Bellomo R, Finfer S, Flabouris A, et al. The objective medical emergency team activation criteria: a case-control study. Resuscitation. 2007 Apr;73(1):62–72. pmid:17241732
  27. 27. Castagna F, McDonnell BJ, Mondellini GM, Gorla R, Pirola F, Landolina M, et al. Twenty-four-hour blood pressure and heart rate variability are reduced in patients on left ventricular assist device support. J Heart Lung Transplant. 2022 Jun;41(6):802–9. pmid:35422348
  28. 28. Heart rate variability: standards of measurement, physiological interpretation and clinical use. Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology. Circulation. 1996 Mar 1;93(5):1043–65. pmid:8598068
  29. 29. Vincent JL, De Backer D. Circulatory shock. N Engl J Med. 2013 Oct 31;369(18):1726–34. pmid:24171518
  30. 30. Bonanno FG. Clinical pathology of the shock syndromes. J Emerg Trauma Shock. 2011 Apr;4(2):233–43. pmid:21769211
  31. 31. Fusaro MV, Becker C, Scurlock C. Evaluating Tele-ICU implementation based on observed and predicted ICU mortality: a systematic review and meta-analysis. Crit Care Med. 2019 Apr;47(4):501–7. pmid:30688718
  32. 32. Cummings BC, Blackmer JM, Motyka JR, Shaltis PA, Bleyle LA, Benjamin KA, et al. External validation and comparison of a general ward deterioration index between diversely different health systems. Crit Care Med. 2023 Jun 1;51(6):775–86. pmid:36927631
  33. 33. Green M, Lander H, Snyder A, Hudson P, Churpek M, Edelson D. Comparison of the Between the Flags calling criteria to the MEWS, NEWS and the electronic Cardiac Arrest Risk Triage (eCART) score for the identification of deteriorating ward patients. Resuscitation. 2018 Feb;123:86–91. pmid:29169912
  34. 34. Singh K, Valley TS, Tang S, Martin J, Agniel D, Kuo M, et al. Evaluating a widely implemented proprietary deterioration index model among hospitalized patients with COVID-19. Ann Am Thorac Soc. 2021 Jul;18(7):1129–37. pmid:33357088
  35. 35. Podell J, Pergakis M, Yang S, Felix R, Parikh G, Chen H, et al. Leveraging continuous vital sign measurements for real-time assessment of autonomic nervous system dysfunction after brain injury: a narrative review of current and future applications. Neurocrit Care. 2022 Aug;37(Suppl 2):206–19. pmid:35411542
  36. 36. Brabrand M, Hallas P, Folkestad L, Lautrup-Larsen CH, Brodersen JB. Measurement of respiratory rate by multiple raters in a clinical setting is unreliable: a cross-sectional simulation study. J Crit Care. 2018 Apr;44:404–6. pmid:29310091
  37. 37. Churpek MM, Snyder A, Twu NM, Edelson DP. Accuracy comparisons between manual and automated respiratory rate for detecting clinical deterioration in ward patients. J Hosp Med. 2018 Jul 1;13(7):486–7. pmid:29394299
  38. 38. Fieselmann JF, Hendryx MS, Helms CM, Wakefield DS. Respiratory rate predicts cardiopulmonary arrest for internal medicine inpatients. J Gen Intern Med. 1993 Jul;8(7):354–60. pmid:8410395
  39. 39. Guechi Y, Pichot A, Frasca D, Rayeh-Pelardy F, Lardeur JY, Mimoz O. Assessment of noninvasive acoustic respiration rate monitoring in patients admitted to an Emergency Department for drug or alcoholic poisoning. J Clin Monit Comput. 2015 Dec;29(6):721–6. pmid:25614223
  40. 40. Latten GHP, Spek M, Muris JWM, Cals JWL, Stassen PM. Accuracy and interobserver-agreement of respiratory rate measurements by healthcare professionals, and its effect on the outcomes of clinical prediction/diagnostic rules. PLoS One. 2019;14(10). pmid:31581207
  41. 41. Subbe CP, Davies RG, Williams E, Rutherford P, Gemmell L. Effect of introducing the Modified Early Warning score on clinical outcomes, cardio-pulmonary arrests and intensive care utilisation in acute medical admissions. Anaesthesia. 2003 Aug;58(8):797–802. pmid:12859475