The CAM-ICU-7 and ICDSC as measures of delirium severity in critically ill adult patients

Background In clinical practice, a dichotomous approach to delirium identification may no longer be relevant when existing delirium screening tools measure a range of scores. The objective of this study was to compare the Confusion Assessment Method for the Intensive Care Unit 7-item (CAM-ICU-7) and the Intensive Care Delirium Screening Checklist (ICDSC) as measures of the spectrum of delirium severity in critically ill adult patients. Methods In this cross-sectional study, 218 patients underwent 641 paired assessments by bedside nurses (ICDSC, as per usual care) and trained research assistants (CAM-ICU-7). Correlation between the CAM-ICU-7 and ICDSC scores was evaluated. Logistic regression was used to explore associations between CAM-ICU-7 or ICDSC score and length of ICU stay and mechanical ventilation (receipt, ≥96 hours). Results Delirium prevalence evaluated by the CAM-ICU-7 and ICDSC were 46.3% (95% CI:39.7–53.0) and 34.4% (95% CI:28.3–41.0). Prevalence of less than clinical threshold symptoms of delirium evaluated by the CAM-ICU-7 (score: 1–2) and ICDSC (score: 1–3) were 30.3% (95%CI:24.5–36.7) and 50.9% (95%CI:44.3–57.6). The CAM-ICU-7 and ICDSC had significant positive correlation (0.58, p<0.001). Agreement between the tools as measures of delirium was moderate (kappa = 0.51) and as measures of less than clinical threshold symptoms of delirium was fair (kappa = 0.21). Less than clinical threshold symptoms of delirium identified by the ICDSC, not CAM-ICU-7, were associated with prolonged length of ICU stay (≥7 days) in patients <65 years of age [Odds Ratio (OR) 9.2, 95% CI:2.5–34.0] and mechanical ventilation (receipt: OR 2.8, 95% CI:1.3–6.4; ≥96 hours: OR 6.6, 95% CI:1.9–22.9), when compared to patients with no delirium. Conclusions The CAM-ICU-7 and ICDSC are measures of the spectrum of delirium severity that are closely correlated. Less than clinical threshold symptoms of delirium measure by the ICDSC is a better predictor of outcomes, when compared with the CAM-ICU-7.

bundle, including routine delirium assessment using the ICDSC once per shift. Metrics for each component are recorded on a bedside critical care clinical information system (eCritical) and are regularly audited to ensure ABCDEF bundle compliance. We followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines for crosssectional studies [18] (Fig 1). We included adults (�18 years of age) with no primary direct brain injury (pre-existing neurological comorbidities [e.g., epilepsy, multiple sclerosis, dementia] was not an exclusion criteria), who could provide consent or surrogate consent, could communicate with the study team (i.e., understand English, no hearing impairment) and were expected to remain in the ICU for at least 24 hours. Patients with a Richmond Agitation- Sedation Scale (RASS) of -4 or -5 or Glasgow Comma Scale of � 9 [19] were excluded. The study was approved by the Conjoint Health Research Ethics Board at the University of Calgary (Reference number: REB16-2060).

Sampling & recruitment
All consecutive, eligible patients admitted to the FMC ICU or their surrogate decision makers were asked by the bedside registered nurse if a research assistant could approach them to discuss a research opportunity. The patient's capacity to provide written, informed consent was decided by the bedside registered nurse. If the patient (or surrogate decision maker) agreed, informed consent was sought, and the patient was enrolled in the study. If a patient regained capacity, consent was sought.

Delirium assessments
The RASS was used to assess a patients' level of sedation or agitation [20]. The RASS is a tool with excellent interrater reliability (k = 0.91, 95% confidence interval (CI) = 0.86-0.95) [20]. Patients with a RASS score of -4 or -5 were not eligible for delirium assessments and, as such, were marked as a missed delirium assessment for that day. Delirium was assessed twice daily in all patients using the CAM-ICU-7 (at 11:00 and 16:00) and ICDSC (at 06:00 and 18:00) for up to five days during their ICU stay. The CAM-ICU-7 and RASS were performed by trained research assistants (KDK, BKR), who were blinded to all bedside registered nurse's delirium and RASS assessments conducted during the five days of data collection. The ICDSC and RASS assessments were performed by trained bedside registered nurses who were also blinded to the study team's delirium and RASS assessments conducted within the first 12 hours of ICU admission.
A comparison between the CAM-ICU-7 and ICDSC as a measurement of the spectrum of delirium severity is demonstrated in Fig 2. To assess the spectrum of delirium severity, less than clinical threshold symptoms of delirium were considered present if a CAM-ICU-7 assessment was negative, but at least one CAM-ICU feature was present (i.e. acute change or fluctuating course, inattention, altered level of consciousness or disorganized thinking) also represented by a CAM-ICU-7 score of 1-2 [22]. For the ICDSC, a patient was considered to have no delirium with a score of 0, less than clinical threshold symptoms of delirium with a score of 1-3 and clinical delirium with a score of 4-8 [5,6].

Other data and clinical outcomes
The patient (or surrogate) completed a demographics questionnaire that included age, education, ethnicity, gender and sex). Clinical characteristics such as severity of illness (Acute Physiology and Chronic Health Evaluation [APACHE] II score upon ICU admission) and organ failure (Sequential Organ Failure Assessment [SOFA] score upon ICU admission) were obtained from eCritical, which is validated for use in research [23]. We identified short term outcomes that were associated with delirium [24] and may be correlated with CAM-ICU-7 or ICDSC scores to assess predictive validity. These short-term outcomes were collected from electronic medical records and included receipt/length of invasive mechanical ventilation during the ICU stay, length of ICU stay and ICU mortality.

Statistical analyses
Descriptive statistics were examined for all study variables. Continuous variables with a normal distribution were presented as mean ± standard deviation [SD]. Correlation between the CAM-ICU-7 and ICDSC scores were estimated by Pearson's correlation coefficient, wherein a Pearson r value of 0.10, 0.30, and 0.50 was interpreted as small, medium, and large effect sizes, respectively [25]. Kappa was calculated as a measure of agreement between the meaning of a Visual representation of the a) CAM-ICU-7 and b) ICDSC as measurements of the spectrum of delirium severity. If a patient exhibits less than clinical symptoms of delirium (i.e., negative CAM-ICU-7 and one of the CAM-ICU features [acute change, fluctuating course, altered level of consciousness, disorganized thinking] or 1-3 items on the ICDSC [altered level of consciousness, inattention, disorientation, hallucination/ delusion/psychosis, psychomotor agitation or retardation, inappropriate mood or speech, sleep wake cycle, fluctuations), less than clinical threshold symptoms of delirium (commonly referred to as subsyndromal delirium) is present (light pink box). A patient is considered to screen positive for delirium if their CAM-ICU-7 score is greater than or equal to three or their ICDSC score is greater than or equal to four (red box).
https://doi.org/10.1371/journal.pone.0242378.g002 CAM-ICU-7 or ICDSC score (i.e., less than clinical threshold symptoms of delirium, delirium), wherein agreement was interpreted as fair (0.21-0.40), moderate (0.41-0.60), substantial (0.61-0.80) and almost perfect (0.81-1.00) [26]. Variables considered potential effect modifiers or confounders were identified a priori, and included age, sex and severity of illness (APA-CHE-II score upon ICU admission). Race was not included as a potential effect modifier or confounder because ICU literature reports no association between race and delirium [27,28]. Logistic regression analysis was used to assess the relationship of prolonged length of ICU stay (�7 days), receipt of invasive mechanical ventilation (ever/never), invasive mechanical ventilation �96 hours [29,30] and ICU mortality with delirium severity. Box plots were used to show median and quartiles of length of ICU stay and length of mechanical ventilation with respect to CAM-ICU-7 and ICDSC score categories. For all analyses, one delirium measure per patient (the most severe CAM-ICU-7 or ICDSC score) was used. All analyses were performed using Stata, version 14.0 (StataCorp, College Station, TX, USA).

Results
From November 2017 to March 2019, 910 patients were screened, 356 patients were eligible and 218 patients were enrolled, with a participation rate of 61.2% (218/356) (Fig 1). The majority of enrolment was completed by patient consent (55.5%, 121/218). The majority of participants were male (59.2%, 129/218), with a mean age of 58.8 years (SD 15.5) ( Table 1) Overall agreement between CAM-ICU-7 and ICDSC as a measurement of delirium (i.e., CAM-ICU-7 score of 3-7 and ICDSC score of 4 or greater) was moderate (kappa = 0.51) ( Table 2). However, the agreement between the two tools when less than clinical threshold symptoms of delirium were present (i.e., CAM-ICU-7 score 1-2 and ICDSC score 1-3) was only fair (kappa = 0.21). The most common ICDSC items identified in patients with less than clinical threshold symptoms of delirium are shown in Table 3. Tables 4 and 5 include the unadjusted (crude), stratified and adjusted odds ratios (ORs) of length of ICU stay associated with CAM-ICU-7 or ICDSC scores, respectively. There was no effect modification or confounding by patient sex, age or APACHE-II score for the CAM-ICU-7. Length of ICU stay was significantly associated with CAM-ICU-7 scores of 3-5 (crude OR 3.7, 95%CI: 1.5-7.5) and 6-7 (crude OR 51.4, 95%CI: 6.5-408.8), compared to patients who did not have delirium during their ICU stay (CAM-ICU-7 score of 0). There was no effect modification or confounding by patient sex or APACHE-II score for the ICDSC. There was effect modification by patient age (p = 0.007) for patients with an ICDSC score of 1-3: the odds ratio for length of ICU stay for patients <65 years of age (n = 76) was 9.2 (95% CI: 2.5-34.0) and for �65 years of age (n = 35) was 0.7 (95% CI: 0.2-2.7), when compared to patients who did not have delirium during their ICU stay (ICDSC score of 0). There was no effect modification or confounding for ICDSC score of 4 to 8 (crude OR 11.1, 95%CI: 5.2-29.2). There appeared to be a similar association between CAM-ICU-7 or ICDSC score and odds of prolonged length of ICU stay wherein an increased score on either tool was associated with increased median length of ICU stay (Fig 3). , when compared to patients with no delirium (CAM-ICU-7 score of 0). There was effect modification by patient sex for receipt of    9), compared to patients who did not have delirium during their ICU stay (ICDSC score of 0). There appeared to be a similar association between length of mechanical ventilation and CAM-ICU-7 score or ICDSC score wherein an increased score on either tool was associated with increased median length of mechanical ventilation (Fig 3).

Mortality
Due to the small number of patients who died in the ICU (n = 7), there were insufficient observations to calculate associations between mortality and CAM-ICU-7 or ICDSC scores.

Discussion
We explored the relationship between the CAM-ICU-7 and ICDSC as measures of the spectrum of delirium severity in critically ill adults. The results of current study suggest the CAM-ICU-7 and ICDSC are significantly correlated as measures of the spectrum of delirium severity. In addition, CAM-ICU-7 and ICDSC scores have a dose-dependent association with prolonged length of ICU stay and mechanical ventilation �96 hours. The association between Table 4. Unadjusted, adjusted and stratified odds ratio estimates associations between prolonged length of ICU stay (�7 days) or mechanical ventilation (ever/ never, �96 hours) and CAM-ICU-7 score. � If there is effect modification, the odds ratio (OR) for the strata where there is significant interaction is reported. If there is no significant interaction or confounding with sex, age or APACHE-II score identified, the interaction is listed as not significant (NS).

Unadjusted
https://doi.org/10.1371/journal.pone.0242378.t004 Table 5. Unadjusted, adjusted and stratified odds ratio estimates associations between prolonged length of ICU stay (�7 days) or mechanical ventilation (ever/ never) and ICDSC score. adverse clinical outcomes differs between the CAM-ICU-7 and ICDSC. For example, less than clinical threshold symptoms of delirium was significantly associated with prolonged length of ICU stay and mechanical ventilation in the ICDSC (not CAM-ICU-7). The agreement between CAM-ICU-7 and ICDSC for a clinical delirium diagnosis has been reported in several studies [7][8][9]11]. In the current study, the kappa was moderate for agreement between the scales for measurement of overall delirium, which is similar to the kappa coefficients (0.50, 0.55) previously reported [9,10], suggesting these tools performed similarly across studies. However, when comparing the ability of these tools to identify less than clinical threshold symptoms of delirium, the agreement for these tools was only fair. When comparing these two tools, different characteristics may explain the variability in prevalence of patients with less than clinical threshold symptoms of delirium (i.e., 66 patients with a CAM-ICU-7 score of 1-2 versus 111 patients with an ICDSC score of 1-3). The CAM-ICU-7 is a point in time assessment of the patient's mentation, in which the assessor asks the patient a series of cognitive questions (3-minutes), two different times during a 12-hour period. In contrast, the ICDSC score is calculated from the bedside registered nurse's observations of the patient during a 12-hour shift, which may identify more symptoms of delirium (e.g., psychomotor agitation or retardation, inappropriate mood or speech, sleep wake cycle and fluctuations which are observed throughout the shift) and thus more patients with an ICDSC score of 1-3. Another key difference is that the tools measure different delirium features. For example, only the ICDSC measures a disturbance in the patient's sleep-wake cycle. Patients admitted to the ICU often experience sleep disturbance [31]. As such, it is expected that many patients would score at least a one on the ICDSC (i.e., less than clinical threshold symptoms of delirium). In the current study, an ICDSC score of one due to sleep-wake cycle disturbance occurred in 21.6% (63/ 292) of ICDSC assessments that identified less than clinical threshold symptoms of delirium. Given the different operating characteristics and measurement of delirium features in the CAM-ICU-7 and ICDSC, similar scores on these tools may not result in similar delirium presentations in each patient.

Unadjusted
The results of the current study have applicability to the most frequently used measures of delirium in critically ill adult patients (CAM-ICU-7 or ICDSC) in clinical practice. In the current study, less than clinical threshold symptoms of delirium are associated with increased length of ICU stay, receipt of mechanical ventilation and mechanical ventilation �96 hours in the ICDSC (not CAM-ICU-7). As such, there is a prognostic significance of less than clinical threshold of delirium symptoms (ICDSC score 1-3) wherein ICU staff should address these symptoms of delirium (with nonpharmacological strategies included in the most recent clinical practice guidelines) to reduce the ICDSC score to 0 (i.e., no delirium) and prevent these adverse clinical events from occurring [32]. These strategies could include targeting the most common ICDSC items identified in patients with less than clinical threshold symptoms of delirium: psychomotor agitation or retardation (pharmacologic and non-pharmacologic treatment of pain and agitation), disorientation (reorientation protocols), sleep wake cycle (assistcontrol ventilation, noise/light reduction) [33].

Strengths and limitations
The current study had several strengths. The study sample was diverse, including critically ill patients with a wide range of critical illness diagnoses from a large, tertiary care medical centre. We collected twice-daily delirium assessments conducted by trained research assistants and by bedside RNs who are regularly audited for delirium assessments as part of an ICU delirium sustainability initiative. The study was co-designed with patients, researchers and clinicians and was registered a priori. The study protocol was tested during a pilot study [34]. Limitations include the cross-sectional nature of data collection, which resulted in a mixture of incident and prevalent delirium cases with no ability to distinguish. Small sample size led to large confidence intervals in subgroups and the inability to analyse the association between delirium score and mortality. Moreover, mortality in the study population is lower than the 9% reported by the Canadian Institute for Health Information [34], which is attributed to recruitment of a patient population with lower morality (less severe illness severity given requirements to be responsive to delirium assessments). The two screening tools were administered by two different raters (CAM-ICU was administered by trained research assistants while the ICDSC required a trained clinician). The inclusion of clinical judgement with the ICDSC assessment may have accounted for differences in delirium detection. The type and dose of sedative agents, antipsychotics or other medications that may have confounded a delirium diagnosis were not measured. The presence of a pre-existing neurologic illness such as dementia was not recorded. There was no difference between median days of delirium by delirium severity for the CAM-ICU and ICDSC. Future studies should explore the association between duration of severity of delirium and patient outcomes.

Conclusions
The current study demonstrated that the CAM-ICU and ICDSC correlate in identification of the spectrum of delirium severity of critically ill adults. The CAM-ICU-7 and ICDSC have fair agreement for identifying less than clinical threshold symptoms of delirium and moderate agreement for identifying delirium. The data suggest that, as the CAM-ICU-7 and ICDSC scores increase, the odds of poor short-term outcomes (length of ICU stay and receipt and length of mechanical ventilation) increase.