Hospitals are increasingly using critical care outreach teams (CCOTs) to respond to patients deteriorating outside intensive care units (ICUs). CCOT staffing is variable across hospitals and optimal team composition is unknown.
To assess whether adding a critical care medicine trained physician assistant (CCM-PA) to a critical care outreach team (CCOT) impacts clinical and process outcomes.
We performed a retrospective study of two cohorts—one with a CCM-PA added to the CCOT (intervention hospital) and one with no staffing change (control hospital)—at two facilities in the same system. All adults in the emergency department and hospital for whom CCOT consultation was requested from October 1, 2012-March 16, 2013 (pre-intervention) and January 5-March 31, 2014 (post-intervention) were included. We performed difference-in-differences analyses comparing pre- to post-intervention periods in the intervention versus control hospitals to assess the impact of adding the CCM-PA to the CCOT.
Measurements and Main Results
Our cohort consisted of 3,099 patients (control hospital: 792 pre- and 595 post-intervention; intervention hospital: 1114 pre- and 839 post-intervention). Intervention hospital patients tended to be younger, with fewer comorbidities, but with similar severity of acute illness. Across both periods, hospital mortality (p = 0.26) and hospital length of stay (p = 0.64) for the intervention vs control hospitals were similar, but time-to-transfer to the ICU was longer for the intervention hospital (13.3–17.0 vs 11.5–11.6 hours, p = 0.006). Using the difference-in-differences approach, we found a 19.2% reduction (95 confidence interval: 6.7%-31.6%, p = 0.002) in the time-to-transfer to the ICU associated with adding the CCM-PA to the CCOT; we found no difference in hospital mortality (p = 0.20) or length of stay (p = 0.52).
Citation: Gershengorn HB, Xu Y, Chan CW, Armony M, Gong MN (2016) The Impact of Adding a Physician Assistant to a Critical Care Outreach Team. PLoS ONE 11(12): e0167959. https://doi.org/10.1371/journal.pone.0167959
Editor: Chiara Lazzeri, Azienda Ospedaliero Universitaria Careggi, ITALY
Received: July 19, 2016; Accepted: November 23, 2016; Published: December 12, 2016
Copyright: © 2016 Gershengorn et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: This work was supported by the National Science Foundation (CMII 1233547) for CWC; National Heart Lung Blood Institute (UO1 HL122998, U01 HL123009, UH2 HL125119) for MNG. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Critical illness can arise anywhere in the hospital, emergency department, or even prior to hospital arrival. Moreover, critical illness is often optimally managed when treatment is initiated early.[1, 2] For these reasons, numerous quality groups advocate for rapid response/medical emergency teams (RR/METs) to attend quickly to patients experiencing clinical deterioration outside the intensive care unit (ICU).[3–5] While data on their impact is mixed,[6–10] these teams are increasingly common across U.S. hospitals and internationally.
Who and how many clinicians should staff these RR/METs to make them most clinically- and cost-effective is not known. Recommendations are that team composition be determined by each “institution’s resources and needs.” Published literature suggests many models—including intensivists, hospitalists, housestaff, non-physician-providers and critical care nurses—are used.[12–16] How one team structure compares to another and the impact of including clinicians with different backgrounds is not well studied. In a single center study of RR/METs led by intensivists versus resident physicians, there was no difference in patients’ progression to cardiac arrest, need for ICU admission, or hospital mortality; yet, still more work needs to be done to understand whether this is the “right” RR/MET structure or if other configurations may be more effective and/or just as effective, but with lower costs.
The role of physician assistants with training/experience in critical care medicine (CCM-PAs) has received some attention in the literature.[17–19] This work has focused primarily on in-ICU staffing[20–22] and has found that employing CCM-PAs as replacements for physicians-in-training can be a safe and effective way to expand the workforce in critical care. Whether replacing an existing team member or adding a CCM-PA to an RR/MET could provide similar benefits or, conversely, might be detrimental is unknown. Specifically, having the CCM-PA take on some of the RR/MET work (e.g. performing procedures, evaluating and managing deteriorating patients awaiting ICU admission, etc.) might reduce delays in ICU admission and, possibly, improve patient outcomes; however, adding a CCM-PA could negatively impact care by delaying assessment by a different provider.
The two tertiary, academic hospitals at Montefiore Medical Center have had a critical care outreach team (CCOT)[11, 23, 24] composed of an intensivist and critical care fellow since 1995. In 2013, a CCM-PA was added to the CCOT at one of the hospitals. Staffing changes are common in healthcare systems, but the impact of such interventions is rarely rigorously evaluated. In this study, we sought to test the hypothesis that adding the CCM-PA would result in better clinical and process (e.g., delays in ICU transfer) outcomes for patients evaluated by the CCOT.
We conducted a retrospective cohort study of all patients seen by the adult CCOT at one of the two academic tertiary care facilities (one with 620 total and 46 critical care beds, the other, 396 total and 33 critical care beds) at Montefiore Medical Center in the Bronx, New York. The larger hospital had 24,878 hospital admissions with an overall hospital mortality of 2.2% in 2014; the smaller, 22,471 admissions and a 1.7% mortality rate. The role of the CCOT in these institutions is to respond to medical emergencies and cardiac arrests outside of the emergency room and ICUs, to evaluate new critical care consultations in the emergency department, on the wards, and in the post-anesthesia care unit, and to provide procedural assistance (e.g., vascular access) throughout the hospital. All patients with acute deterioration or unplanned ICU transfer require CCOT evaluation prior to ICU admission; most surgical patients enter the ICU in the immediate post-operative period without CCOT evaluation. CCOTs began in both institutions in 1995 and were staffed since inception by a board certified intensivist and critical care fellow 24 hours/day, 7 days/week. Starting on March 17, 2013, a CCM-PA was gradually added (first during the daytime on weekdays, then weekends, and then overnight) to the CCOT at the larger hospital (intervention hospital) with complete PA coverage 24 hours/day, 7 days/week by December 16, 2013; to allow for a run-in, we used January 5, 2014 as the initiation date for the post-period analyses. During this time, Montefiore Medical Center employed 37 CCM-PAs; a different CCM-PA per 12.5 hour shift staffed the CCOT. The PA’s role included assisting in new patient evaluations, stabilizing/monitoring deteriorating patients not yet in the ICU, and performing procedures (e.g., central venous catheterizations). Prior to the addition of the CCM-PA, all of these tasks were performed by the board certified intensivist and critical care fellow who comprised the CCOT. During the same time, there was no addition of a PA to the control hospital because of limited resources. No other changes were made to the staffing structures or roles/responsibilities of these CCOTs during this time.
Our cohort consisted of all patients for whom CCOT consultation was requested in each of the two hospitals from October 1, 2012 –March 16, 2013 (pre-intervention) or January 5 –March 31, 2014 (post-intervention); patients consulted upon from March 17, 2013 –January 4, 2014 were excluded as there was incomplete CCM-PA staffing during this period (Fig 1). Patients were excluded if consultations were clearly requested solely for procedural assistance. Montefiore Medical Center has a separate pediatric CCOT system, thus, the vast majority of CCOT consultations were on adults (≥21 years-old); however, patients were not specifically excluded based on age.
CCM-PA: Critical Care Medicine Physician Assistant; CCOT: Critical Care Outreach Team.
Clinical data were available from the electronic medical record and were accessed using Clinical Looking Glass. Patient-level characteristics included demographics (age, race/ethnicity, gender, insurance provider), chronic illness burden (determined by Elixhauser comorbidities), primary admission diagnosis category (determined by International Classification of Diseases, 9th Edition coding on hospital discharge), severity of illness (determined by Laboratory-based Acute Physiology Score, LAPS, and Sequential Organ Failure Assessment score, SOFA) at the time of CCOT evaluation, need for ICU admission, goals of care (full care including resuscitation/intubation if needed versus orders not to resuscitate and/or intubate) at the time of CCOT evaluation, timing of hospital admission (weekend versus weekday), and timing of CCOT evaluation (night—7pm-6:59am versus day—7am-6:59pm, day of week, month). Outcomes were time-to-transfer to the ICU (time from the initial request for CCOT evaluation to when the first set of vital signs was recorded in the ICU, TtTICU) for patients admitted to the ICU, hospital mortality, and hospital length of stay (LOS).
Standard summary statistics were used to compare baseline characteristics and unadjusted outcomes between patients admitted to the intervention and control hospitals (both pre- and post-intervention). A difference-in-differences approach was used to evaluate the impact of the intervention (introducing the CCM-PA to the CCOT) after adjustment for potential confounders.[29–32] This methodology uses a multivariable regression modelling technique with an interaction term for “time period” (pre- versus post-intervention) and “hospital” (intervention versus control) to allow assessment of the independent association of the intervention with an outcome. In contrast to more standard before-after studies, the use of the control hospital enables control for unobservable temporal trends in patient severity that might impact outcomes.
Two underlying assumptions are central to the difference-in-difference technique: that (1) no interventions other than that being studied occurred during the study period to one hospital differently than the other and (2) trends in outcomes for the control and intervention hospitals were similar pre-intervention (Fig 2). As aforementioned, no other changes were made to the CCOT at either hospital during this period. To test the second assumption, we used multivariable regression modeling for only patients admitted during the pre-intervention period; the model included the date of CCOT evaluation, the hospital (control versus intervention), and all other patient-level covariates. An insignificant association of hospital (control versus intervention) and outcome would indicate similar baseline trends for that outcome.
Assumptions of the methodology include: (1) single intervention as depicted by the ⇧ and (2) same baseline trends in outcome as demonstrated by parallel outcomes prior to the intervention. The difference-in-differences is equal to the difference between the intervention and control hospitals pre-intervention (differencepre-intervention) minus the difference between the intervention and control hospitals post-intervention (differencepost-intervention).
Separate multivariable difference-in-difference regression models were constructed to evaluate the association of the CCM-PA CCOT intervention and TtTICU, hospital mortality, and hospital LOS; TtTICU and hospital LOS were logarithmically transformed. In addition to the inclusion of hospital (intervention versus control), time period (pre- versus post-intervention), and their interaction term, all measured patient-level characteristics were included in each model as covariates. Finally, four additional models were constructed for TtTICU using three-way interactions (hospital*time period*“stratified groups” with “stratified groups” defined separately by age </≥ 65, Elixhauser comorbidities </≥ 2, diagnosis of infectious disease vs not, and diagnosis of respiratory illness vs not) to assess whether adding the CCM-PA to the CCOT affected subgroups differentially.
Statistical analyses were performed using R 3.2.3 (The R Foundation for Statistical Computing, Vienna, Austria) and Microsoft Excel (Microsoft, Redmond, WA). Patient information was de-identified prior to analysis. Institutional Review Board approval was obtained from Albert Einstein College of Medicine (#2014–3190).
Our cohort consisted of 3,099 patients (control hospital: 792 pre- and 482 post-intervention; intervention hospital: 1114 pre- and 711 post-intervention, Table 1). Patients evaluated by the CCOT at the intervention hospital tended to be younger (62.7 years-old pre- and 61.3 post-intervention vs 67.4 and 66.3 in the control hospital, p<0.01), more commonly African-American (38.8% pre- and 38.0% post-intervention vs 30.2% and 30.8%, p<0.01), and more likely to have Medicare/Medicaid insurance (64.5% pre- and 65.8% post-intervention vs 59.0% and 60.5%, p<0.01). They tended to have fewer comorbidities (Elixhauser comorbidities: 4.3 pre- and 4.3 post- intervention vs 4.5 pre- and 4.6 post-, p<0.01) and lower LAPS (59.2 pre- and 55.4 post-intervention vs 60.1 pre- and 59.9 post-, p = 0.037), but higher SOFA (3.9 pre- and 3.7 post-intervention vs 3.7 pre- and 3.6 post-, p<0.01) at the time of CCOT consultation. While LAPS and SOFA scores both measure acuity, their use of vital signs and laboratory data differ.
Analysis of pre-intervention trends revealed no difference (0.2% per day change in TtTICU, 95% confidence interval: -0.001%–0.3%) associated with being at the intervention versus control hospital. Similarly, there was no difference in the percent change for: (1) odds of hospital mortality (0.2% (-0.3%–0.8%)) and (2) hospital LOS (0.06% (-0.02%–0.13%)). These results support the assumption necessary for difference-in-difference analyses—that trends in outcomes for the control and intervention hospitals were similar pre-intervention.
For those patients transferred to the ICU, unadjusted TtTICU was longer in the intervention hospital (mean±sd: 17.0±15.0 hours pre- and 13.3±11.6 post-intervention vs 11.6±11.3 pre- and 11.6±8.7 post-, p = 0.006); notably, TtTICU was shorter in the post- vs pre-intervention period (p = 0.004) in the intervention but not the control hospital (p = 0.90, Fig 3). The difference-in-difference analysis demonstrated a significant 19.2% reduction in the adjusted TtTICU (95% confidence interval: 6.7%-31.6% reduction, p = 0.002) associated with adding the CCM-PA to the CCOT (Table 2).
(A) Time-to-transfer. (B) Hospital mortality. (C) Hospital length of stay.
There was no difference in unadjusted hospital mortality between hospitals (p = 0.26) with no change pre- to post-intervention in either center (intervention hospital: 21.8% pre- vs 19.9% post-intervention, p = 0.30; control hospital: 21.2% pre- vs 22.9% post-; p = 0.47). Moreover, we found no association between CCM-PA addition to the CCOT and adjusted mortality using the difference-in-difference methodology (p = 0.20). Similarly, there was no difference in unadjusted hospital LOS between the hospitals (p = 0.64) with no change pre- to post-intervention in either center (intervention hospital, mean±sd: 18.6±28.0 days pre- vs 18.6±22.8 post-intervention, p = 0.97; control hospital: 14.2±17.4 pre- vs 14.7±17.1 post-, p = 0.61). There was also no association between CCM-PA addition to the CCOT and adjusted hospital LOS using the difference-in-difference approach (p = 0.52).
Our results were consistent across subgroups. There was no association between CCM-PA addition to the CCOT and adjusted hospital mortality or LOS for patients admitted to the ICU (S1 Table). Adding the CCM-PA to the CCOT affected adjusted TtTICU, hospital mortality, and hospital LOS similarly across subgroups of age, Elixhauser comorbidity burden, and infectious disease and respiratory admitting diagnoses (Fig 4).
Introducing a CCM-PA to the CCOT at our institution was associated with a statistically significant and clinically meaningful reduction in the TtTICU of 19.2%; or, without casemix adjustment, a 3.7 hour reduction after adding the CCM-PA (with no change at the control hospital). We were unable to identify any impact on hospital mortality or LOS, however. Interestingly, the impact on TtTICU was experienced similarly by older and younger patients, patients with high and low burden of chronic disease, and patients with different admitting diagnoses.
Our finding that adding a CCM-PA to the CCOT decreased TtTICU is novel; several factors likely drive it. First, the CCM-PA may have been the first CCOT member to evaluate patients ultimately in need of the ICU while other members were busy. The literature suggests that well-trained PAs are able to identify patients needing ICU admission; when asked to quantify patients’ likelihood of clinical deterioration, nurse practitioners and PAs anticipated patient instability as well as physicians (area under the curve for “patient acuity rating” of 0.80 for nurse practitioners/PAs vs 0.69 for medicine residents and 0.88 for attendings). Thus, having these patients assessed by the CCM-PA earlier than they would be by other CCOT members could facilitate ICU transfer. Second, the CCM-PA may have been evaluating less sick patients and/or performing procedures which freed the attending or fellow to more expeditiously evaluate and activate transfers for patients in need of ICU. Finally, the CCM-PA may have identified clinical deteriorations in patients not initially thought to need the ICU on close follow-up which, had the CCM-PA not been available, would have gone unnoticed for longer. As TtTICU is defined as the time from initial request for CCOT evaluation to time of ICU arrival, this quicker identification of clinical deterioration would improve TtTICU.
Our finding that CCM-PA addition to the CCOT can improve care processes without compromising outcome is consistent with published literature. While CCOTs and RR/METs are differentially staffed across institutions,[12–16] there is little data on the relative benefits of one team composition over another. The only study to directly compare staffing structures was conducted at a single academic hospital where RR/METs were led by intensivists during the daytime and medical residents overnight/on weekends. Patients seen by the RR/MET during the day (versus overnight/on weekends) had similar rates of progression to cardiac arrest (1.8% vs 2.4%, p = 0.4), ICU transfer (57% vs 57%, p = 0.82), and hospital mortality (27% vs 26%, p = 0.64). Put simply, it didn’t matter who staffed the team. Furthermore, as aforementioned, nurse practitioners and PAs are as able as physicians to predict clinical deterioration. These studies suggest, therefore, we can view adding a CCM-PA to our 2-person CCOT as, simply, a 50% increase in qualified personnel and capacity. Moreover, this capacity increase is achieved at approximately one-third the cost of adding an intensivist (were expansion of the team to be achieved in this way)—median CCM-PA annual salary is $90,000-$110,000 versus $275,000-$300,000 for an intensivist.[34, 35] As described by Jones et al., there is a dose-response curve for RR/METs activations—the more times they evaluate patients (the more patients they consult on) the better their impact on outcomes. Thus, it is reasonable and consistent with published literature to observe that adding a CCM-PA to our CCOT and expanding its reach improves the CCOT’s performance. Furthermore, adding the CCM-PA may be cost effective, in terms of capacity-per-dollar.
That we did not identify an association of adding a CCM-PA to the CCOT and hospital mortality can be explained in two ways—either: (1) there is truly no impact on mortality or (2) we did not have adequate power to detect a difference which actually exists. Multiple reviews and meta-analyses have been done reaching conflicting conclusions on the impact of RR/METs on hospital mortality.[6, 8–10] It is possible, therefore, that there is truly no such impact of this intervention. Notably, however, multiple studies have reported on the negative association of delays in ICU admission (prolonged TtTICU) and mortality—in the emergency department, throughout the hospital, post-operatively, and from hospital wards . In two, a delayed transfer was defined as TtTICU > 6 hours.[37, 39] We found adding the CCM-PA reduced TtTICU by an average of 3.7 hours suggesting that a subset of patients had a reduction in TtTICU of >6 hours which may also result in reducing in odds of mortality. It reasonable to suppose, therefore, that given the reduced TtTICU we saw with adding the CCM-PA to our CCOT, with a larger sample, this reduction may translate into a survival benefit. In fact, we did observe a non-significant 23.8% reduction in the odds of mortality associated with adding the CCM-PA.
Our study is the first to assess the changing composition of a CCOT and to evaluate the impact of this change on TtTICU. It has limitations, however. As a single center study, more work is necessary to determine whether and how our results generalize to other institutions, especially given the variability in CCOT structure and function across hospitals as well as differences in non-CCOT resources available to identify and care for unstable patients. Given our retrospective study design, we were unable to describe the precise role the CCM-PA played for each individual patient. Their overall responsibilities were diverse; which role(s) was paramount during each encounter is not known. Moreover, we are unable to determine whether the impact of the CCM-PA was the addition of a PA in specific or, simply, the addition of a third CCOT member. Teasing out this difference will enable better determination of whether adding a CCM-PA is indeed cost-effective. If the benefits are due to the addition of a third CCOT member, cost savings related to salary differences between CCM-PAs and intensivists/other providers alone may result in cost-effectiveness; but, a full cost-effectiveness analysis is needed before this impact can be known. Moreover, if the PA brings a specific unique beneficial contribution which an intensivist would not, they may represent not only be a less costly, but also a more effective, addition. Finally, we investigated the impact of adding to the CCOT a PA with experience working in the ICU; we cannot say whether our results would be similar had PAs with different proficiencies been utilized. These limitations are balanced by our study’s strengths. First, we had data for a control hospital with similar patient population, staffing patterns, and trends in outcome pre-intervention. This allowed use of the difference-in-differences methodology to account for secular changes in practice and, thus, better evaluation of the specific impact of our intervention. This approach allows us to better control for temporal trends as well unobserved biases common across the two hospitals when compared to other single institution pre/post-implementation studies which do not have a control hospital available. Second, to address seasonal variability of critical illness both in terms of patient diagnoses/severity of illness and staffing competencies at academic centers, we compared outcomes between pre- and post-intervention periods which occurred during similar times of year.
Adding a CCM-PA to our CCOT (staffed initially by an intensivist and critical care fellow) resulted in a significant improvement in patient flow without an observable impact on outcomes. Improving patient flow is, on its own, a meaningful endpoint for hospital administrators (who must report flow metrics), clinicians (who may prefer to care for critically ill patients in the ICU setting), and patients/families (who may exhibit anxiety associated with delays). Understanding how adding the CCM-PA improves flow, however, remains to be explored—does he/she simply represent an increase in capacity for the CCOT or is there something unique he/she provides to expedite throughput? Moreover, a major benefit of CCOTs would be their ability to prevent the need for ICU admission; it remains to be examined whether adding a CCM-PA to the team may assist in this goal. Critical care is a specialty in which the importance of multidisciplinary care is known. How best to utilize multidisciplinary team members in a setting like a CCOT—perhaps by targeting certain patient subgroups—is unclear. As we expand the utilization and reach of RR/METs and CCOTs, we must carefully evaluate how best to construct them to achieve their stated goals.
S1 Table. Multivariate Difference-in-Difference Analysis for Patients Admitted to the Intensive Care Unit*.
* all covariates listed in Table 1 were included in the multivariable models; only intervention-related variables and those with p<0.05 are provided for simplicity. † Time-to-Transfer values are identical to those in Table 2 as Time-to-Transfer includes only, by definition, patients admitted to the intensive care unit. ‡ Age <90 years compared to reference of age ≥90. § Modeled as linear predictors of outcome.
- Conceptualization: HBG YX CWC MA MNG.
- Data curation: YX HBG.
- Formal analysis: YX HBG.
- Funding acquisition: CWC MNG.
- Investigation: YX HBG.
- Methodology: HBG YX CWC MA MNG.
- Project administration: CWC MA MNG.
- Resources: CWC MA MNG.
- Software: YX HBG.
- Supervision: CWC MA MNG.
- Visualization: HBG YX CWC MA MNG.
- Writing – original draft: HBG YX CWC MA MNG.
- Writing – review & editing: HBG YX CWC MA MNG.
- 1. Puymirat E, Caudron J, Steg PG, Lemesle G, Cottin Y, Coste P, et al. Prognostic impact of non-compliance with guidelines-recommended times to reperfusion therapy in ST-elevation myocardial infarction. The FAST-MI 2010 registry. Eur Heart J Acute Cardiovasc Care. 2015.
- 2. Saver JL, Fonarow GC, Smith EE, Reeves MJ, Grau-Sepulveda MV, Pan W, et al. Time to treatment with intravenous tissue plasminogen activator and outcome from acute ischemic stroke. JAMA. 2013;309(23):2480–8. pmid:23780461
- 3. Institute for Healthcare Improvement: Rapid Response Teams 2015 [cited 2015 November 3]. http://www.ihi.org/topics/rapidresponseteams/pages/default.aspx.
- 4. Agency for Healthcare Research and Quality: Rapid Response Team: U.S. Department of Health & Human Services; 2011 [cited 2015 November 3]. http://www.guideline.gov/content.aspx?id=34271#Section420.
- 5. Health Care Protocol: Rapid Response Team, 4th Edition2011 November 3, 2015. https://www.icsi.org/_asset/8snj28/RRT.pdf.
- 6. McNeill G, Bryden D. Do either early warning systems or emergency response teams improve hospital patient survival? A systematic review. Resuscitation. 2013;84(12):1652–67. pmid:23962485
- 7. McGaughey J, Alderdice F, Fowler R, Kapila A, Mayhew A, Moutray M. Outreach and EarlyWarning Systems (EWS) for the prevention of Intensive Care admission and death of critically ill adult patients on general hospital wards (Review). Cochrane Database of Systemic Reviews. 2007.
- 8. Laurens NH, Dwyer TA. The effect of medical emergency teams on patient outcome: a review of the literature. Int J Nurs Pract. 2010;16(6):533–44. pmid:21129105
- 9. Winters BD, Weaver SJ, Pfoh ER, Yang T, Pham JC, Dy SM. Rapid-response systems as a patient safety strategy: a systematic review. Ann Intern Med. 2013;158(5 Pt 2):417–25. pmid:23460099
- 10. Chan PS, Jain R, Nallmothu BK, Berg RA, Sasson C. Rapid Response Teams: A Systematic Review and Meta-analysis. Arch Intern Med. 2010;170(1):18–26. pmid:20065195
- 11. Agency for Healthcare Research and Quality: Patient Safety Primers—Rapid Response Systems [updated October 2012; cited 2013 December 31]. http://psnet.ahrq.gov/primer.aspx?primerID=4.
- 12. Cretikos MA, Parr MJ. The Medical Emergency Team: 21st century critical care. Minerva Anestesiol. 2005;71(6):259–63. pmid:15886586
- 13. Morris DS, Schweickert W, Holena D, Handzel R, Sims C, Pascual JL, et al. Differences in Outcomes Between ICU Attending and Senior Resident Physician Led Medical Emergency Team Responses. Resuscitation. 2012.
- 14. Rothberg MB, Belforti R, Fitzgerald J, Friderici J, Keyes M. Four years' experience with a hospitalist-led medical emergency team: an interrupted time series. J Hosp Med. 2012;7(2):98–103. pmid:21998088
- 15. Repasky TM, Pfeil C. Experienced critical care nurse-led rapid response teams rescue patients on in-patient units. J Emerg Nurs. 2005;31(4):376–9. pmid:16126105
- 16. Kapu AN, Wheeler AP, Lee B. Addition of acute care nurse practitioners to medical and surgical rapid response teams: a pilot project. Crit Care Nurse. 2014;34(1):51–9. pmid:24488890
- 17. Kleinpell R, Ely E, Grabenkort R. Nurse practitioners and physician assistants in the intensive care unit: an evidence-based review. Crit Care Med. 2008;36(10):2888–97. pmid:18766097
- 18. Gershengorn HB, Johnson MP, Factor P. The use of nonphysician providers in adult intensive care units. Am J Respir Crit Care Med. 2012;185(6):600–5. pmid:22135345
- 19. Garland A, Gershengorn HB. Staffing in ICUs: physicians and alternative staffing models. Chest. 2013;143(1):214–21. pmid:23276844
- 20. Dubaybo BA, Samson MK, Carlson RW. The role of physician-assistants in critical care units. Chest. 1991;99(1):89–91. pmid:1670630
- 21. Gershengorn HB, Wunsch H, Wahab R, Leaf D, Brodie D, Li G, et al. Impact of Non-Physician Staffing on Outcomes in a Medical Intensive Care Unit. Chest. 2011.
- 22. Kawar E, DiGiovine B. MICU care delivered by PAs versus residents: do PAs measure up? JAAPA. 2011;24(1):36–41. pmid:21261146
- 23. Abella Álvarez A, Torrejón Pérez I, Enciso Calderón V, Hermosa Gelbard C, Sicilia Urban JJ, Ruiz Grinspan M, et al. ICU without walls project. Effect of the early detection of patients at risk. Med Intensiva. 2013;37(1):12–8. pmid:23059055
- 24. Niven DJ, Bastos JF, Stelfox HT. Critical Care Transition Programs and the Risk of Readmission or Death After Discharge From an ICU: A Systematic Review and Meta-Analysis*. Crit Care Med. 2014;42(1):179–87. pmid:23989177
- 25. Montefiore Medical Center: Clinical Looking Glass [cited 2015 May 15]. http://exploreclg.montefiore.org/.
- 26. Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. Med Care. 1998;36(1):8–27. pmid:9431328
- 27. Escobar GJ, Greene JD, Scheirer P, Gardner MN, Draper D, Kipnis P. Risk-adjusting hospital inpatient mortality using automated inpatient, outpatient, and laboratory databases. Med Care. 2008;46(3):232–9. pmid:18388836
- 28. Vincent JL, Moreno R, Takala J, Willatts S, De Mendonça A, Bruining H, et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med. 1996;22(7):707–10. pmid:8844239
- 29. Ashenfelter O, Card D. Using the Longitudinal Structure of Earnings to Estimate the Effect of Training Programs. The Review of Economics and Statistics. 1985;67(4):648–60.
- 30. Abadie A. Semiparametric Difference-in-Differences Estimators. Review of Economic Studies. 2005;72(1):1–19.
- 31. Angrist JD, Pischke J-S. Mostly Harmless Econometrics: An Empiricist's Companion, 1st Edition. Princeton, New Jersey: Princeton University Press; 2009.
- 32. Dimick JB, Ryan AM. Methods for evaluating changes in health care policy: the difference-in-differences approach. JAMA. 2014;312(22):2401–2. pmid:25490331
- 33. Edelson DP, Retzer E, Weidman EK, Woodruff J, Davis AM, Minsky BD, et al. Patient acuity rating: quantifying clinical judgment regarding inpatient stability. J Hosp Med. 2011;6(8):475–9. pmid:21853529
- 34. 2013 AAPA Annual Survey Report. 2014.
- 35. Society of Critical Care Medicine. Compensation of Critical Care Professionals, Third Edition: Society of Critical Care Medicine; 2014.
- 36. Jones D, Bellomo R, DeVita MA. Effectiveness of the Medical Emergency Team: the importance of dose. Crit Care. 2009;13(5):313. pmid:19825203
- 37. Chalfin DB, Trzeciak S, Likourezos A, Baumann BM, Dellinger RP, group D-Es. Impact of delayed transfer of critically ill patients from the emergency department to the intensive care unit. Crit Care Med. 2007;35(6):1477–83. pmid:17440421
- 38. Cardoso LT, Grion CM, Matsuo T, Anami EH, Kauss IA, Seko L, et al. Impact of delayed admission to intensive care units on mortality of critically ill patients: a cohort study. Crit Care. 2011;15(1):R28. pmid:21244671
- 39. Bing-Hua YU. Delayed admission to intensive care unit for critically surgical patients is associated with increased mortality. Am J Surg. 2014;208(2):268–74. pmid:24480235
- 40. Harris S, Singer M, Rowan K, Sanderson C. Delay to admission to critical care and mortality among deteriorating ward patients in UK hospitals: a multicentre, prospective, observational cohort study. Lancet. 2015;385 Suppl 1:S40.