A Multi-Center Prospective Derivation and Validation of a Clinical Prediction Tool for Severe Clostridium difficile Infection

Background and Aims Prediction of severe clinical outcomes in Clostridium difficile infection (CDI) is important to inform management decisions for optimum patient care. Currently, treatment recommendations for CDI vary based on disease severity but validated methods to predict severe disease are lacking. The aim of the study was to derive and validate a clinical prediction tool for severe outcomes in CDI. Methods A cohort totaling 638 patients with CDI was prospectively studied at three tertiary care clinical sites (Boston, Dublin and Houston). The clinical prediction rule (CPR) was developed by multivariate logistic regression analysis using the Boston cohort and the performance of this model was then evaluated in the combined Houston and Dublin cohorts. Results The CPR included the following three binary variables: age ≥ 65 years, peak serum creatinine ≥2 mg/dL and peak peripheral blood leukocyte count of ≥20,000 cells/μL. The Clostridium difficile severity score (CDSS) correctly classified 76.5% (95% CI: 70.87-81.31) and 72.5% (95% CI: 67.52-76.91) of patients in the derivation and validation cohorts, respectively. In the validation cohort, CDSS scores of 0, 1, 2 or 3 were associated with severe clinical outcomes of CDI in 4.7%, 13.8%, 33.3% and 40.0% of cases respectively. Conclusions We prospectively derived and validated a clinical prediction rule for severe CDI that is simple, reliable and accurate and can be used to identify high-risk patients most likely to benefit from measures to prevent complications of CDI.


Introduction
Clostridium difficile is a gram-positive anaerobic bacterium that is an important human pathogen causing antibiotic associated diarrhea and pseudomembranous colitis [1,2]. C. difficile infection (CDI) is highly prevalent in hospitals and nursing homes where patients frequently receive antibiotics and is now a leading cause of hospital-associated infection [1]. The overall incidence of CDI has increased dramatically in the United States, Canada and parts of Europe in recent years [1,[3][4][5]. Disease severity has also increased with increasingly virulent strains of C. difficile leading to markedly increased case fatality rates [6]. Nosocomial CDI has been estimated to more than quadruple the cost of otherwise matched hospitalizations, totaling up to $3 billion/year in hospital costs in the United States [7][8][9] The clinical outcomes of CDI range from symptomless carriage, to mild diarrhea, to fulminant pseudomembranous colitis and death [1,10]. Several factors appear to influence the clinical outcome of CDI including the virulence of the infecting strain, the general health of the host and the host immune response [6,[11][12][13]. There are recent data to indicate that oral vancomycin is superior to metronidazole for treatment of severe CDI [14]. Hence, recent guidelines recommend using oral vancomycin as first-line therapy in patients presenting with severe CDI [1,15]. These guidelines also acknowledge the need for prospectively validated severity scores for CDI as presented in this study. Thus, it is important to establish clinical prediction tools for severe clinical outcomes in CDI in order to inform early patient management decisions, to improve quality of care to patients suffering from CDI and to facilitate the development and implementation of new interventions to improve clinical outcomes in severe disease. The goal of this study was to develop and validate a simple, easy to apply, clinical prediction rule that could be used at the time of CDI diagnosis to identify patients at high risk for adverse outcomes of the disease.

Patient Populations and Definitions
CDI was defined as diarrhea, coupled with a positive stool assay for Clostridium difficile toxin A or toxin B (evaluated in all cases by enzyme-immunoassay) and not attributed other causes. Diarrhea was defined as a change in bowel habit with 3 or more unformed bowel movements per day for at least 2 days. The primary outcome of interest was "severe CDI" which was defined as a case of CDI leading to any one of the following outcomes: death attributable to CDI or with CDI as a contributing factor; intensive care unit (ICU) admission attributable to CDI or with CDI as a contributing factor; toxic megacolon or colectomy attributable to CDI. The relatedness of ICU admission or death to CDI was determined by chart review performed by two independent physician investigators. In the case of discordant determinations a third physician provided the deciding judgment. All of the severe outcomes evaluated occurred during the same hospital admission when CDI was diagnosed. Our rationale was that severe complications of an acute CDI were likely to occur quite quickly whereas negative outcomes that transpired days or weeks after hospital discharge were unlikely to be related to the index CDI. Patients with CDI were studied at three sites: i) Beth Israel Deaconess Medical Center (BIDMC), Boston, Massachusetts, ii) Mater Misericordiae University Hospital and St Vincent's University Hospital, Dublin, Ireland and iii) Baylor St. Luke's Medical Center, Houston, Texas. The data consisted of clinical information and clinical outcomes data for all patients.

Ethics Statement
The study was approved by the Committee on Clinical Investigations at BIDMC in Boston (local protocol number 2010P-000302), the Committee for the Protection of Research Subjects at the University of Houston (local protocol number CPHS 13101-01), and the Mater Misericordiae University Hospital Ethics Committee (local protocol number 1/378/856). The data presented in this paper from the Houston and Boston sites used information available from patient medical records and as such the protocols at these sites were approved by their respective IRBs with a waiver for informed consent. Written informed consent was obtained from the subjects included in the Dublin cohort as they also participated in a separate larger local study.

Data collection
Demographic data including age, gender and race as well as relevant laboratory studies including peripheral white blood cell counts and serum creatinine were recorded. Baseline characteristics that were examined included severity of co-morbid conditions (determined by the Charlson comorbidity score), seriousness of underlying illness (determined by the modified Horn disease severity index), immunosuppression (congenital immunodeficiency, HIV or AIDS, active malignancy other than basal or squamous cell skin cancer, immunosuppressive medication post organ transplantation, use of systemic corticosteroids [10 mg of oral prednisone or equivalent per day]), chemotherapy agents or other immunosuppressive medications (including anti-TNF antibody therapy, azathioprine, 6-mercaptopurine, cyclosporine or methotrexate within 8 weeks), history of prior CDI, recent use of antibiotics and current acid antisecretory medications [12,[16][17][18][19][20].

Clinical Prediction Rule and Statistical Analyses
Initial evaluation of clinical factors associated with severe outcomes of CDI was performed in the Boston cohort. All adult patients with CDI hospitalized at BIDMC between December 2004 and Jan 2006 were eligible for study entry. Factors in the Boston cohort found to be associated with severe CDI on univariate analysis (Table 1) were used to inform a multivariate logistic regression model ( Table 2). Evaluation of the data from Dublin and Texas indicated some differences between the three sites in the incidence of severe CDI and in the relative frequency of specific events leading to designation as severe CDI (Table 3). For instance, the Irish cohort had relatively low rates of ICU admission compared to the Boston and Texas sites.
The Boston cohort was used to derive a clinical prediction rule for severe outcomes of CDI and the combined Dublin and Texas cohorts were used to evaluate the performance of the predication rule. Initial multivariable logistic regression analysis in the derivation cohort suggested age 65 years, WBC 20,000 cells/μL and serum creatinine 2 mg/dl as the main independent variables associated with severe CDI. As the β coefficients for each variable were relatively low (0.87 to 2.09) the presence or absence of each risk factor was made bivariate with one or no points for the presence of each specific risk factor. Risk scores for each patient were then correlated by simple summation of points for all predictors present. For analysis of sensitivity, specificity and positive and negative predictive values, patients were divided into highrisk and low-risk groups based on the presence of two or more risk factors for severe CDI. All statistical analyses were performed using the SPSS software system, version 17.0 (SPSS Inc, Chicago, IL).

Patient Characteristics
A total of 638 patients with CDI from three clinical sites were recruited into our study. We prospectively identified 263 consecutive patients with CDI at the Boston site from December 2004 to January 2006 and recorded their clinical information. Similar data were collected prospectively for 150 patients with CDI at the Dublin site from November 2007 to June 2009 and for 225 subjects with CDI at the Houston site from January 2006 to August 2010. Forty two subjects lacked essential clinical data leaving a total of 596 evaluable subjects. Overall 53.0% of subjects were female; gender distribution was similar at all three sites ( Table 3). The patients from Dublin and Houston were similar with respect to age while the subjects from Dublin were slightly older (Table 3). A total of 120 patients (18.8%) met the criteria for severe clinical outcomes of CDI (Table 3); 51 out of 263 (19.4%) patients from Boston, 21 out of 150 (14.0%) patients from Dublin and 48 out of 225 (21.3%) subjects from Houston. ICU admission was less frequent in the Dublin patients (1.3%) compared to Boston (12.2%) and Houston (13.8%). Megacolon/colectomy was more frequent in the Boston patient (3.4%) compared to Dublin (0.7%) and Houston (0.9%) patients. Death related to CDI occurred in 4.2% of the patients in Boston compared to 6.7% of the patients in Dublin and 8.0% of the patients in Houston.

Development of a Clinical Prediction Rule for Severe CDI
We first examined patient characteristics and clinical data for the 263 CDI patients in Boston. Univariate analysis identified age, peak peripheral blood white blood cells (WBC) (evaluated within the time period from 5 days before to 2 days after the diagnostic stool sample was obtained) and peak serum creatinine (evaluated within the same 7 day time period) as being significantly higher in CDI patients with severe disease when compared to the non-severe CDI group (Table 1). The nadir of the calculated glomerular filtration rate was also significantly associated with severe CDI but the final clinical prediction rule was not substantially improved by using GFR and so the more simple measure of serum creatinine was used in preference. The other variables examined were not significantly associated with severe CDI. Based on these findings we generated a clinical prediction model for severe CDI outcomes as shown in Table 2. The β coefficients for each of the three predictors (age 65 years, peak serum creatinine 2 mg/dl and peak WBC 20 x 10 3 cells/μL) were similar and so each was assigned a score value of one. As illustrated in Fig 1, upper panel, in the derivation group, patients with higher CDI severity scores showed a greater probability to develop severe clinical outcomes of CDI. Individuals' scores of 0, 1, 2 or 3 had a risk of severe clinical outcomes of CDI of 7.1%, 12.1%, 40.7% and 58.3% respectively (Table 4).
Using a dichotomized score, patients in the derivation group with a low CDI severity score (0 or 1) had a 10.5% risk of severe outcomes as compared to 43.7% in those with a high severity score (2 or 3). The sensitivity, specificity, positive predictive value and negative predictive value of a high CDI severity score (2 or 3) are presented in Table 5. The overall diagnostic accuracy of a high (2 or 3) or a low (0 or 1) CDI severity score in predicting severe versus non-severe CDI clinical outcomes was 76.5% in the derivation group.

Validation of the Clinical Prediction Rule
In order to perform a formal validation of the clinical prediction rule for risk of severe clinical outcomes in CDI, we used combined Houston and Dublin cohorts as the validation cohort. When the prediction rule was applied to the validation cohort the correlation between the prediction score and the clinical outcome of severe CDI was again clearly evident (Fig 1, lower  panel). In the validation group, individuals' risk prediction scores of 0, 1, 2 or 3 were associated with severe clinical outcomes of CDI in 4.7%, 13.8%, 33.3% and 40.0% of cases respectively (Table 4).
Using a dichotomized score, patients in the validation group with a low CDI severity score (0 or 1) had a 12.2% risk of severe outcomes as compared to 34.3% in those with a high severity score (2 or 3). The sensitivity, specificity, positive predictive value, negative predictive value, positive likelihood ratio and negative likelihood ratio of a high CDI severity score (2 or 3) are  presented in Table 5. The overall diagnostic accuracy of a high (2 or 3) or a low (0 or 1) CDI severity score in predicting severe versus non-severe CDI clinical outcomes was 72.5% in the validation group.

The Performance of the Severe CDI Risk Score
When sorted by risk scores, the prediction score identified groups of patients with increasing probability of severe outcomes of CDI as illustrated in Fig 2. In the entire cohort of 596 individuals those with a CDI severity score of 0 had a risk of severe clinical outcomes of 6.7%, individuals with a score of 1 had a risk of severe clinical outcomes of 16.3%, those with a score of 2 had a risk of severe clinical outcomes of 34.9% and a score of 3 was associated with a 46.9% risk of severe clinical outcomes of CDI. Using a dichotomized scoring system of lower (0 or 1) versus higher risk (2 or 3) the incidence of severe CDI clinical outcomes were 11.5% and 38.2% respectively. When the CDI severity score was applied to the patients according to their study site those with higher scores (2 or 3) were at substantially greater risk for severe clinical outcomes from CDI when compared to those with lower scores (0 or 1) at each of the three sites (Fig 3). Patients with a low CDI severity score of 0 or 1 had a risk of severe CDI of 10.6% in Boston, 7.9% in Dublin and 14.6% in Houston. Conversely, patients with a CDI severity risk score of 2 or 3 had a higher probability of developing severe CDI at each site: 43.7% in Boston, 25% in Dublin and 40.7% in Houston (Fig 3).

Discussion
Clostridium difficile infection (CDI) is a common cause of nosocomial diarrhea that is associated with substantial morbidity, mortality and healthcare costs. First-line antibiotic treatment for mild or moderately severe CDI is metronidazole. However, oral vancomycin is recommended for severe CDI. Some patients require intensive care unit admission, and colectomy is necessary for others [1,21]. Mortality rates from CDI in the United States increased from 5.7 per million population in 1999 to 23.7 per million in 2004 [22]. Several new therapeutic agents are being developed for CDI. Fidaxomicin is a new, macrocyclic antibiotic, recently approved for use in CDI and is associated with lower rates of recurrent disease as compared to oral vancomycin [23,24]. Monoclonal antibodies directed against C. difficile toxins have also been found to be effective in preventing CDI recurrences [25]. It is becoming increasingly important to identify patients with a greater likelihood for severe disease who would be most likely to benefit from Clinical prediction rules (CPRs) can help to address these aforementioned challenges in CDI management and, accordingly, several CPRs for CDI complications have been proposed over the years; but none has gained widespread clinical acceptance. A systematic review published in 2012 that specifically addressed CPRs for poor outcomes in CDI concluded that the available CPRs lack weighing variables and strong validation thereby rendering their quality suboptimal and utility debatable [26]. As the authors state: "Evidence-based tools developed through appropriate prospective cohorts would be more valuable for clinicians than empirically-developed CPRs" [26]. The current study addressed these limitations and as such is positioned to become a valuable addition to the field.
In this study we prospectively identified, using multivariate analysis, risk factors identified at the time of CDI diagnosis that independently predict later severe clinical outcomes, or complications, of the disease. Using these risk factors we developed a Clostridium difficile severity score (CDSS) that uses readily available clinical data to predict severe outcomes of CDI. We then validated the CDSS in a separate group of CDI patients. Our study is, to our knowledge, the largest prospective, multi-center study to develop and validate a clinical prediction rule for severe clinical outcomes of CDI reported to date. A total of 638 CDI patients from three institutions were included in our study. Objective markers for severe complications of CDI were used (colectomy, admission to an intensive care unit or death from CDI or with CDI as a contributor). Our study indicates that three simple clinical criteria (age greater than 65 years, peak serum creatinine greater than 2 mg/dL and peak peripheral blood leukocyte count 20,000 cells/μL) can be used to identify patients at greatest risk for severe CDI complications. At initial diagnosis, most patients with CDI present with similar symptoms that do not correlate with the subsequent clinical course of their disease. The CDSS can be determined easily at the time of diagnosis of CDI and alert clinicians to a greater risk for serious complications indicating the need for more aggressive management e.g. use of oral vancomycin as the first-line agent and discontinuing all other non-essential antimicrobial drugs. The CDSS can also be valuable in developing, testing and applying new treatment approaches to avert severe complications of CDI.
A recent systematic review identified 13 studies on the derivation of a CPR for CDI: two on recurrences, five on complications, five on mortality and one on response to treatment [26]. The authors noted that there were a number of limitations in these studies i.e. heterogeneity in the variables used, small sample size, and only a few studies performed multivariate analyses to adjust for confounders. In that review, the authors found that only three studies (Lungulescu et al., Bhangu et al. and Zilberberg et al.) dealing with CDI severity had a good quality of derivation methodology [27][28][29].
Lungulescu et al. created a CDI severity index (CSI) score based on four risk factors that were identified by univariate analysis: history of malignancy, white blood cell count at admission 20,000/dL, blood albumin <3.0mg/dL and creatinine at admission >1.5 fold the baseline value. One point was assigned to each of the risk factors. A CSI score with a cut-off value of 2 had a sensitivity and specificity of 82% and 65%, respectively. Our study reaffirms the clinical significance of having an elevated leukocyte count and creatinine at the time of CDI diagnosis in predicting severe CDI. In our study data were collected prospectively-in keeping with the general principal that predictive variables should be collected prospectively-and therefore more accurately than in prior studies, in a process established specifically for the development and the validation of clinical rules [30,31]. The study by Lungulescu et al. is also limited by not using multivariate analyses and lacking a validation cohort.
Bhangu et al. created a score based on five risk factors that were identified by a multi-variate analysis: age 80 years, clinically severe disease, white blood cell count 20,000/dL or CRP 150 mg/dL, blood urea nitrogen > 15 mmol/L and albumin < = 2.0mg/dL. One point was assigned to each of the risk factors. Point counts of 0-1, 2-3 and 4-5 were associated with mortality rates of 22%, 55% and 89% respectively. Bhangu et al. collected the data prospectively. However, they did not have a validation cohort and did not report performance parameters of scores.
Zilberberg et al. created a score based on five risk factors that were identified by a multi-variate analysis: age 75 y, no respiratory disease, septic shock, lack of leukocytosis, APACHE II score 20. Absence of chronic respiratory disease (R), age 75+ yrs (A), septic shock (S) and Acute Physiology and Chronic Health Evaluation II score 20+ (A) comprised the RASA score, whose receiver operating characteristics was 0.740; 95% Confidence Interval was 0.663-0.817. The RASA score was put forth as a prediction tool for 30-day mortality and clinical management in an elderly population afflicted by CDI. Older age was one of the variables in Zilberberg's score initially derived to predict mortality among the elderly, but the weight given to age was potentially over-estimated by being also included within the APACHE II score. Other limitations of the study include retrospective data collection and the absence of a validation cohort.
Horn's index, a severity score based on underlying clinical illness, has also been shown to be to be a useful method for identifying CDI patients at high risk of poor clinical and economic outcomes [20]. A recent prospective study from Texas demonstrated that patients with Horn's index scores of 3 or 4 had a significantly longer hospital stay (mean 33.4 [standard deviation, SD 33.3] days) than patients with scores of 1 or 2 (mean 15.1 [SD 16.2] days, P = 0.001) [12]. The study was smaller (85 subjects) and validation was not performed. Also, Horn's index is a subjective assessment by a physician; it is not known whether clinicians from different regions would assign the same Horn's score to a specific patient. Although our current model addresses several weaknesses of previous studies there are also some methodological limitations. We recognize that dichotomizing continuous variables sacrifices information, but we opted to do so to increase the ease of clinical application of our prediction tool. It would be ideal to validate this model in a large external observational cohort, but unfortunately, we did not have access to data from another prospective cohort study of CDI of similar size. Therefore, we opted to conduct internal validation using the split-sample method. The predictive model we developed and validated is consistent with the risk factors identified in previous studies and updating or validating the model externally would involve minor calibration, if any. We believe that the risk score we developed is generalizable to other CDI cohorts. Indeed, our model worked well despite differences in the healthcare systems, study populations and the frequency of severe outcomes among the Boston, Houston and Ireland cohorts included in our analysis. Another potential limitation is that we did not perform strain typing of C. difficile isolates in this study. Some, but not all, studies have found that infection with specific strains of C. difficile (such as ribotypes 027, 078 or 244) may be associated with more severe outcomes. [5,6,18,32] In previous studies 25% of isolates from hospital patients at the Boston site were ribotype 027 (BI/NAP-1), 24% were ribotype 027 and 6% were 078 in Houston whereas in Dublin 26% were 027 and 14% were 078. Ribotype 244 was not identified. [18,33] However, strain type information is rarely available to clinicians at the time of diagnosis and treatment of C. difficile infection. Hence, we did not consider it a practical or valuable inclusion for a clinical prediction tool.
In summary, we derived and validated a predication rule for severe CDI and showed that the risk of severe CDI can be reliably predicted using a simple risk-scoring system. Consistent with prior studies, advanced age, elevated leukocytes and creatinine were associated with severe clinical outcomes of CDI. This score can be helpful to practitioners to identify patients at high risk of developing severe CDI outcomes and to provide aggressive treatment with vancomycin early on in their clinical course. The score will also help in risk stratification for targeting novel interventions appropriately. Further, prospective validation will be important to consolidate and refine the prediction rule.