Figures
Abstract
Introduction
Delirium is a prevalent condition in patients admitted to intensive care units (ICU) associated with worse outcomes. The principal aim of the present study was compare the agreement between two tools for delirium assessment in medical and surgical patients admitted to the ICU.
Methods
Consecutive adult surgical and medical patients admitted to the ICU for more than 24 hours between March 2009 and September 2010 were included. Delirium was evaluated twice a day using the Intensive Care Delirium Screening Checklist (ICDSC) and Confusion Assessment Method adapted to the Intensive Care Unit (CAM-ICU). The kappa (k) and AC1 coefficients were calculated as a measure of agreement between the CAM-ICU and ICDSC.
Results
A total of 595 patients were enrolled in the study. There were 69 (12%) emergency surgical, 207 (35%) elective surgical and 319 (54%) medical patients. Delirium incidence evaluated by the ICDSC, but not by the CAM-ICU, was similar among the three groups. Overall agreement between CAM-ICU and ICDSC was moderate (k = 0.5) to substantial (AC1 = 0.71). In medical patients the agreement between the two instruments was moderate (k = 0.53) to substantial (AC1 = 0.76). The agreement between the two tools in emergency surgical patients was also moderate (k = 0.53) to substantial (AC1 = 0.68). In elective surgical patients the agreement between the two instruments was low (k = 0.42) to substantial (AC1 = 0.74).Agreement rates seemed to be influenced by disease severity. The agreement rate in the general ICU population with APACHE II = <14 was k = 0.57 and AC1 = 0.81, compared to k = 0.44 and AC1 = 0.59, in patients with more severe disease. This was even more different when the need for mechanical ventilation was used as a surrogate of disease severity.
Citation: Fagundes JAdO, Tomasi CD, Giombelli VR, Alves SC, Macedo RCd, Topanotti MFL, et al. (2012) CAM-ICU and ICDSC Agreement in Medical and Surgical ICU Patients Is Influenced by Disease Severity. PLoS ONE 7(11): e51010. https://doi.org/10.1371/journal.pone.0051010
Editor: Gabriel Alejandro de Erausquin, University of South Florida, United States of America
Received: May 18, 2012; Accepted: October 29, 2012; Published: November 30, 2012
Copyright: © 2012 Fagundes et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This research was supported by NENASC project (PRONEX program CNPq / FAPESC) and INCT-TM. CR, JQ, MS and FDP are CNPq Research Fellows. CDT is holder of a FAPESC studentship. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.
Competing interests: Co-author JS is a PLOS ONE Editorial Board member. This does not alter the authors' adherence to all the PLOS ONE policies on sharing data and materials.
Introduction
Delirium is a prevalent medical condition associated with worse outcomes in patients admitted to intensive care units (ICU) [1], [2]. It is defined by some key features as changes in mental status characterized by a reduced awareness of the environment and a disturbance in attention [3], [4]. Other symptoms as hallucinations, disorientation or temporary memory dysfunction can also occur [5]–[7]. The incidence of delirium in ICU patients ranges from 19% to 87% [8], and postoperative delirium (POD) incidence ranges from 11% to 42% depending on the study population [8], [9]. Such large differences in delirium incidence among the studies can be ascribed to several factors including patients’ characteristics (e.g. type of ICU admission, older age, severity of illness), as well as to the diagnostic tool used to diagnose delirium. The most frequently used instruments to diagnose delirium in the ICU setting are the Confusion Assessment Method for the Intensive Care Unit - CAM-ICU [10] and the Intensive Care Delirium Screening Checklist - ICDSC [11]. The CAM-ICU, adapted from the Confusion Assessment Method, was introduced for the use in mechanically ventilated patients [12]. Originally validated by Ely et al, the CAM-ICU showed a high sensitivity (93%) and specificity (89%) in diagnosing delirium [10]. Lin et al [13] subsequently validated the CAM-ICU in a cohort of mechanically ventilated medical patients and reported similar results. Interestingly, Bergeron and et al validated the ICDSC in ICU patients demonstrating a higher sensitivity as compared to the CAM-ICU (99%) but a lower specificity (64%) [11]. However, the ICDSC is the only tool whose ability to detect subsyndromal delirium has been studied [14].
Direct comparisons between these tools were performed with discordant results [15]–[18]. In addition, to our knowledge, no study has compared these tools specifically in different subgroups as medical and surgical patients. Therefore, the aim of the present study was evaluate the agreement between the CAM-ICU and the ICDSC for delirium diagnosis in different subgroups of ICU patients.
Methods
Ethics Statement
The local institutional review board (Research with Humans Ethics Committee of the São José Hospital) approved the present study and written informed consent was obtained from all patients or their legal representatives.
Study Design, Setting and Patients’ Selection
This was a prospective cohort study performed between March 2009 and September 2010. Consecutive adult (older than 18 years) patients admitted to a 20-bed medical-surgical ICU at a tertiary teaching Hospital for more than 24 hours were included. We excluded readmissions, moribund patients and those with a Richmond Agitation and Sedation Scale (RASS) [19] score equal to-4 or -5 during the entire study period. Clinical data were recorded daily until ICU discharge. Vital status at ICU and hospital discharge were obtained in all patients.
Delirium Assessment
Delirium was assessed in all patients using both the CAM-ICU and the ICDSC, twice a day (by 08∶00 AM and 02∶00 PM) during their ICU stay. Delirium assessments were performed by investigators fully trained in the use of both scales (CDT, VRG, SCA, TCM, MFLT).
The ICDSC evaluates the level of consciousness, inattention, disorientation, hallucinations, psychomotor activity, speech or mood disturbance, sleep disturbance, and fluctuation of symptoms [11]. According to this instrument, patients were considered to have delirium when at least four of the above mentioned eight items were deviant, and subsydromal delirium was diagnosed in patients with scores between 1 and 3 [11], [17].
According to the CAM-ICU, patients had a diagnosis of delirium when an acute onset of mental status change or a fluctuating course and inattention were accompanied by either disorganized thinking or an altered level of consciousness [10]. The level of consciousness was assessed with the RASS [17], ranging from -5 (unarousable) to +4 (combative).
Statistical Analyses
Standard descriptive statistics were used to characterize the study population. Continuous variables with normal distribution were presented as mean ± standard deviation and compared by t-Student test or one way ANOVA, as appropriate. Continuous variables with a non-normal distribution were reported as median (25%–75% interquartile range) and compared using Mann-Whitney U test or Kruskal-Wallis test, as appropriate. Categorical variables were presented as absolute numbers (frequency percentages) and analyzed by Chi-square test or Fisher exact test, as appropriate. The diagnostic value of the CAM-ICU and ICDSC were described using 2 X 2 tables. The kappa (k) and AC1 coefficients were calculated, and their correspondent 95% confidence intervals, as a measure of agreement between the CAM-ICU and ICDSC. Agreement was graded as slight (0–0.20), fair (0.21–0.40), as moderate (0.41–0.60), substantial (0.61–0.80) or almost perfect (0.81–1.0). A two-tailed p-value < 0.05 was considered statistically significant. All the analyses were performed with SPSS for Windows, version 17.0, and R-projetc software version 2.15.1.
Results
During the study period 813 patients were assessed for eligibility, of whom 218 (27%) were excluded (Figure 1). Thus, 595 patients were enrolled into the study, and grouped according to the type of admission into emergency surgical (n = 69, 12%), elective surgical (n = 207, 35%) and medical (n = 319, 54%). The median age and gender distribution were similar among the three groups (Table 1). As expected, the APACHE II (Acute Physiology and Chronic Health disease Classification System II) and SOFA (Sequential Organ Failure Assessment score) scores were significantly higher in emergency surgical and medical patients as compared to elective surgical patients (Table 1). Delirium incidence ranged from 10% to 34.0%, depending on the group of patients and tool used for the diagnosis (Tables 1 and 2). The frequencies of delirium were comparable among the three groups using the ICDSC (Table 1). In contrast, when evaluated by the CAM-ICU, medical patients had a higher incidence as compared to the other groups (Table 1). Overall agreement between CAM-ICU and ICDSC was moderate (k = 0.5) to substantial (AC1 = 0.71) (Table 2). In medical patients, concordant results were found in 258 (81%) patients and the agreement between the two instruments was moderate (k = 0.53) to substantial (AC1 = 0.76) (Table 2). The agreement between the two tools in emergency surgical patients was also moderate (k = 0.53) to substantial (AC1 = 0.68). No patient presented with positive CAM-ICU and negative ICDSC. Concordant results were found in 58 (84%) patients (Table 2). In elective surgical patients, concordant positive results were found in 19 (9%) patients, while concordant negative results were found in 151 (73%) patients. Discordant results were observed in 37 (18%) patients. The agreement between the two instruments was low (k = 0.42) to substantial (AC1 = 0.74) in these patients (Table 2).
ICU = Intensive Care Unit.
Since disease severity could contribute to these differences it was determined the agreement in the subset of patients with APACHE II ≤ 14 and >14 (based on the median of APACHE II score in the sample). The agreement rate in the general ICU population with APACHE II = <14 was k = 0.57 and AC1 = 0.81, compared to k = 0.44 and AC1 = 0.59 in patients with more severe disease. In emergency surgical patients the agreement between the two scales assessed both by kappa and AC1 was similar (Table 3), but it differs in medical patients (k = 0.65 and AC1 = 0.82 compared to k = 0.43 and AC1 = 0.52, respectively). In elective surgical patients, kappa agreement rate was 0.44 and AC1 was 0.80 compared to k = 0.40 and AC1 = 0.65 in patients scored by APACHE II in ≤14 and > 14, respectively. We also attempted to evaluate the effect of severity of illness using the need for mechanical ventilation as a surrogate to identify a more severe group of patients, observing even more different agreements rates (Table 4).
Discussion
In the present study, we demonstrated that the agreement rate between CAM-ICU and ICDSC is in general moderate, but varies with the type of ICU admission and severity of disease.
The agreement between scales for delirium diagnosis was the object of a few studies [16]–[18] but, to our knowledge, we provide the first investigation attempting to analyze in separate the agreement between CAM-ICU and ICDSC in medical and surgical patients (elective and emergency surgery) admitted to the ICU and stratified by severity of illness. In a study comparing the agreement between these scales in general ICU patients, kappa’s coefficients ranged from 0.65 to 0.92 [16]. We had previously observed a kappa agreement rate of 0.59 both in a single center study and a multicenter study [17], [18]. Interestingly, even between subsets of surgical patients the agreement rate varies. There are some different characteristics observed in patients that can be related to this variability. Medical and emergency / urgency patients are usually more severely ill at ICU admission as compared to elective surgery patients. In addition, these patients are more prone to use sedation and we can suppose that these differences can interfere in the agreement rate between CAM-ICU and ICDSC. In fact, according to our data it seems that the observed differences in the agreement rates between medical and urgency surgery compared to elective surgery are mainly related to disease severity.
We demonstrated that in the present study population the incidence of delirium did not differ significantly between medical (26%), elective (35%) and emergency surgery (28%) when delirium was evaluated by the ICDSC. In contrast, when evaluated by the CAM-ICU there was a higher delirium incidence in medical patients (20%) when compared to elective surgical patients (10%) and emergency surgical (13%) patients. Several studies had shown that the occurrence of delirium in postoperative patients is common [20]–[22], as it is in the general ICU patients [23]. Patients who were exposed to major surgeries or emergency surgery and developed delirium had more postoperative complications than the patients who never develop delirium [20]–[22]. In addition, medical patients also presented worse outcomes when develop delirium [12]. Nevertheless, delirium is probably under diagnosed [24]. Thus it seems that the low positivity of CAM-ICU in surgical patients indicates that, for this subset of patients, the ICDSC can be a better screening tool. These differences in the performance of the scales also seem to be related to disease severity. In patients presenting with less severe disease delirium positivity was similar in both medical and surgical patients independent on the diagnosis tool that was used. In contrast, in patients presenting with APACHE II score higher than 14 the positivity of CAM-ICU, but not ICDSC, was significantly more frequent in the medical group. The application of CAM-ICU, differently from ICDSC, is more dependent on the interaction between the interviewer and patient, thus is an active diagnosis tool. It is plausible to suggest that as more severely ill, more difficult the interaction between the interviewer and patient (mainly in patients presenting with RASS -3) leading to more difficult tool application. In contrast, ICDSC seems to be more subjective when compared to CAM-ICU, suggesting that its higher positivity is associated a low specificity of delirium diagnosis.
Some limitations of our study must be pointed out. Despite the large sample size this is a single center study. Second, we do not include evaluation of delirium using gold-standard diagnosis by the DSM-IV criteria, thus we can not evaluate sensitivity and specificity of these tool nor ascertain that the differences on CAM-ICU and ICDSC positivity really reflects differences on diagnosis of delirium. This is minimized by the results from a multicenter study demonstrating similar kappa values when comparing CAM-ICU and ICDSC [18]. Third, no statistical analyses were done to compare agreement rates, nor if disease severity is an independent risk factor for agreement of the two delirium assessment tools. We had tried to assess this, but the regression for the concordance that we had performed have, in general, poor discriminative capacity. In addition, kappa analyses are subject to “kappa paradox” which in turn limits the interpretation of agreement through its estimation and a formal (statistic) comparison between two kappa values. We tried to minimize this performing two different agreement analyses, the kappa and the AC1. In addition, from the clinical point of view there is no meaning to determine the variables associated with the agreement between the scales, but we just need to know which tool works better for a determined patient.
Conclusion
In conclusion, agreement rates between CAM-ICU and ICDSC may vary between different groups of ICU patients and seems to be affected by disease severity.
Key Messages
- The agreement rate between CAM-ICU and ICDSC is in general moderate, but varies depending on the type of ICU admission and severity of disease.
- Medical and emergency / urgency patients have more severe disease at ICU admission, and they more prone to use sedation and this can interfere in the agreement rate between CAM-ICU and ICDSC.
Author Contributions
Conceived and designed the experiments: JAOF FDP CR CDT. Performed the experiments: VRG SCA RCM MFLT MLUB. Analyzed the data: CR FDP CDT PEAAB MS JS. Wrote the paper: CR FDP MS JS CDT.
References
- 1. Morandi A, Pandharipande P, Trabucchi M, Rozzini R, Mistraletti G, et al. (2008) Understanding international differences in terminology for delirium and other types of acute brain dysfunction in critically ill patients. Intensive Care Med 34: 1907–1915.
- 2. Ely EW, Gautam S, Margolin R, Francis J, May L, et al. (2001) The impact of delirium in the intensive care unit on hospital length of stay. Intensive Care Med 27: 1892–1900.
- 3.
American Psychiatric Association (2000) Diagnostic and statistical manual of mental disorders American Psychiatric Press, Washington.
- 4. Inouye SK (2006) Delirium in older persons. N Engl J Med 354: 1157–1165.
- 5. Deiner S, Silverstein JH (2009) Postoperative delirium and cognitive dysfunction. Br J Anaesth 103 (Suppl 1)41–46.
- 6. Siddiqi N, House AO, Holmes JD (2006) Occurrence and outcome of delirium in medical in-patients; a systematic literature review. Age and Ageing 35: 350–364.
- 7. Inouye SK, Charpentier PA (1996) Precipitating factors for delirium in hospitalized elderly patients: predictive model and interrelationship with baseline vulnerability. JAMA 275: 852–857.
- 8. Ouimet S, Kavanagh BP, Gottfried SB, Skrobik Y (2007) Incidence, risk factors and consequences of ICU delirium. Intensive Care Med 33: 66–73.
- 9. Gunther ML, Morandi A, Ely EW (2008) Pathophysiology of delirium in the intensive care unit. Crit Care Clin 24 (Suppl 1)45–65.
- 10. Ely EW, Margolin R, Francis J, May L, Truman B, et al. (2001) Evaluation of delirium in critically ill patients: validation of the Confusion Assessment Method for the Intensive Care Unit (CAM-ICU). Crit Care Med 29: 1370–1379.
- 11. Bergeron N, Dubois MJ, Dumont M, Dial S, Skrobik Y (2001) Intensive Care Delirium Screening Checklist: evaluation of a new screening tool. Intensive Care Med 27: 859–864.
- 12. Ely EW, Shintani A, Truman B, Speroff T, Gordon SM, et al. (2004) Delirium as a predictor of mortality in mechanically ventilated patients in the intensive care unit. JAMA 291: 1753–1762.
- 13. Lin SM, Liu CY, Wang CH, Lin HC, Huang CD, et al. (2004) The impact of delirium on the survival of mechanically ventilated patients. Crit Care Med 32: 2254–2259.
- 14. Cole M, McCusker J, Dendukuri N, Han L (2003) The prognostic significance of subsyndromal delirium in elderly medical inpatients. J Am Geriatr Soc 51: 754–760.
- 15. van Eijk MM, van Marum RJ, Klijn IA, de Wit N, Kesecioglu J, et al. (2009) Comparison of delirium assessment tools in a mixed intensive care unit. Crit Care Med 37: 1881–1885.
- 16. Plaschke K, von Haken R, Scholz M, Engelhardt R, Brobeil A, et al. (2008) Comparison of the confusion assessment method for the intensive care unit (CAM-ICU) with the Intensive Care Delirium Screening Checklist (ICDSC) for delirium in critical care patients gives high agreement rate(s). Intensive Care Med 34: 431–436.
- 17. Tomasi CD, Grandi C, Salluh J, Soares M, Giombelli VR, et al. (2011) Comparison of CAM-ICU and ICDSC for the detection of delirium in critically ill patients focusing on relevant clinical outcomes. J Crit Care 27(2): 212–217.
- 18. Gusmao-Flores D, Salluh JI, Dal-Pizzol F, Ritter C, Tomasi CD, et al. (2011) The validity and reliability of the portuguese versions of three tools used to diagnose delirium in critically ill patients. Clinics 66: 1917–1922.
- 19. Sessler CN, Gosnell MS, Grap MJ, Brophy GM, O’Neal PV, et al. (2002) The Richmond Agitation–Sedation Scale: Validity and Reliability in Adult Intensive Care Unit Patients. Am J Respir Crit Care Med 166: 1338–1344.
- 20.
Koebrugge B, van Wensen RJ, Bosscha K, Dautzenberg PL, Koning OH (2010) Delirium after emergency/elective open and endovascular aortoiliac surgery at a surgical ward with a high-standard delirium care protocol. Vascular (Suppl 5): 279–287.
- 21. Olin K, Eriksdotter-Jönhagen M, Jansson A, Herrington MK, Kristiansson M, et al. (2005) Postoperative delirium in elderly patients after major abdominal surgery. Br J Surg 92(12) 1559–1564: 9.
- 22. Mu DL, Wang DX, Li LH, Shan GJ, Li J, et al. (2010) High serum cortisol level is associated with increased risk of delirium after coronary artery bypass graft surgery: a prospective cohort study. Crit Care 2010 (Suppl 6)R238.
- 23. Salluh JI, Soares M, Teles JM, Ceraso D, Raimondi N, et al. (2010) Delirium epidemiology in critical care (DECCA): an international study. Crit Care 14: R210.
- 24. Salluh JI, Dal-Pizzol F, Mello PV, Friedman G, Silva E, et al. (2010) Delirium recognition and sedation practices in critically ill patients: A survey on the attitudes of 1015 Brazilian critical care physicians. J Crit Care 24: 556–562.