Development of an organ failure score in acute liver failure for transplant selection and identification of patients at high risk of futility

Introduction King’s College Hospital criteria are currently used to select liver transplant candidates in acetaminophen-related acute liver failure (ALF). Although widely accepted, they show a poor sensitivity in predicting pre-transplant mortality and cannot predict the outcome after surgery. In this study we aimed to develop a new prognostic score that can allow patient selection for liver transplantation more appropriately and identify patients at high risk of futile transplantation. Methods We analysed consecutive patients admitted to the Royal Free and Beaujon Hospitals between 1990 and 2015. Clinical and laboratory data at admission were collected. Predictors of 3-month mortality in the non-transplanted patients admitted to the Royal Free Hospital were used to develop the new score, which was then validated against the Beaujon cohort. The Beaujon-transplanted group was also used to assess the ability of the new score in identifying patients at high risk of transplant futility. Results 152 patients were included of who 44 were transplanted. SOFA, CLIF-C OF and CLIF-ACLF scores were the best predictors of 3-month mortality among non-transplanted patients. CLIF-C OF score and high dosages of norepinephrine requirement were the only significant predictors of 3-month mortality in the non-transplanted patients, and therefore were included in the ALF-OFs score. In non-transplanted patients, ALF-OFs showed good performance in both exploratory (AUC = 0.89; sensitivity = 82.6%; specificity = 89.5%) and the validation cohort (AUC = 0.988; sensitivity = 100%; specificity = 92.3%). ALF-OFs score was also able to identify patients at high risk of transplant futility (AUC = 0.917; sensitivity = 100%; specificity = 79.2%). Conclusion ALF-OFs is a new prognostic score in acetaminophen-related ALF that can predict both the need for liver transplant and high risk of transplant futility, improving candidate selection for liver transplantation.

Introduction Acetaminophen overdose (APAP-OD) is the most frequent cause of acute liver failure (ALF) in Western countries [1]. ALF is a life-threatening condition characterized by rapid severe liver injury and hepatic encephalopathy in patients without pre-existing liver disease. The clinical presentation is characterized by abnormal liver biochemical values, coagulopathy, decline in mental function, peripheral vasodilatation, features of the systemic inflammatory response syndrome and ultimately multi-organ failure (MOF) [2]. The period of active injury in acetaminophen overdose can be self-limiting and displays a hyperacute pattern in majority of patients; most of them recover with medical management alone including N-acetyl cysteine [3]. However, in patients who continue to deteriorate, an emergency liver transplantation (LT) is the only life-saving option and survival is inversely related to the time period elapsed between listing and the procurement of an organ.
The decision-making process for LT is currently based upon the King's College Hospital criteria (KCH), which includes a set of parameters dedicated specifically to acetaminopheninduced ALF [4,5]. However, in a recent meta-analysis, KCH criteria, while showing a specificity of 95%, was associated with a very poor sensitivity (58%) in predicting LT-free mortality [6]. Several alternative prognostic scores have been developed with the aim to optimise sensitivity further while retaining specificity [7][8][9]. In addition to the difficulties faced with accurately identifying suitable LT candidates in ALF, it is important to recognise that the LT procedure itself is associated with high peri-and post-operative mortality and long-term complications, and requires life-long treatment with immunosuppression [10]. In the context of organ shortages and potential complications of LT, it is imperative that determining suitability of LT in ALF patients should also take in to account the probability of survival after LT in order to optimize organ allocation thus avoiding "futile" transplantation.
The primary aim of this study was to develop a prognostic score that would accurately predict the 3-month mortality in acetaminophen induced ALF (with or without LT) thus avoiding futile emergency LTs. Four different assessment strategies were used to address this hypothesis: 1) study best predictors of poor prognosis amongst the commonly used current scoring systems applied to patients with critical liver diseases; 2) to develop a new score using the best existing scores and additional clinical and biochemical variables, and validate its prognostic accuracy in an external cohort; 3) to characterize the pre-transplant features that may indicate high risk of futile LT; 4) to validate the accuracy of the new score in predicting futility of LT in an independent cohort.

Study population and statistical analysis
We analysed all consecutive patients admitted with acetaminophen related ALF between 1990 and 2015 to the Intensive care unit of Royal Free Hospital (RFH), London (United Kingdom) and to the Liver Intensive Care Unit of Beaujon Hospital (BJH), Clichy (France). ALF was defined as the presence of severe liver injury with onset of hepatic encephalopathy (HE) within 12 weeks of the first symptoms [11]. Patients with pre-existing liver disease were excluded. Clinical and laboratory data were collected at the time of admission as well as the use of mechanical ventilation support, vasopressors and continuous renal replacement therapy. The study endpoint was all-cause mortality during the first 3 months after the admission. Patients were listed for liver transplantation according to KCH criteria [4] (the modified version including lactate levels was applied from 2002) [5]. Data for this study was obtained through archived patient notes in the hospital and the follow up data retrieved through a combination of follow up clinic notes, patient's general physicians and direct telephone contact with patients themselves. This database is updated at regular intervals and has been analysed for other purposes previously. The research and development department at the Royal Free Hospital where this project was undertaken, have defined the study as a clinical audit and service evaluation project with no requirement for formal ethics approval. The data was fully anonymised and the need for consent was waived by the R&D department.
Continuous parametric variables were expressed as mean ± standard deviation and were compared using Student t-test. Non-parametric variables were showed as median and range, and compared with Kruskal-Wallis Test. Categorical variables were compared using the Chisquared test. SPSS software package (version 20.0, SPSS Inc., Chicago, Ill, United States) and Medcalc1 (version14.8.1, MedCalc Software bvba) were used for statistical analysis.

Performance of existing scores
The principal liver-specific and general intensive care scores were calculated at time of admission. Those who received an emergency liver transplant (LT) were analysed separately from non-transplant (NOLT) patients. The KCH criteria were considered met in APAP-OD patients with pH<7.3 or lactate>3.5mmol/L following adequate resuscitation or the presence of following three features: international normalized ratio of Protrombin time (INR)>6.5, serum creatinine >3.4 mg/dl and a grade 3 or 4 hepatic encephalopathy based on West Haven criteria [12]. The Chronic Liver Failure Consortium Organ failure score (CLIF-C OF), with a range from 0 to 18, evaluates the failure of six organ systems (liver, kidney, brain, coagulation, circulation and respiratory system) taking into account the serum bilirubin, serum creatinine, INR, mean arterial blood pressure, PaO 2 and fractional inspired concentration of oxygen (FiO 2 ), PaO 2 /FiO 2 ratio and the use of renal replacement therapy, vasopressors and invasive mechanical ventilation. Chronic Liver Failure Consortium Acute on Chronic Liver Failure (CLIF-C ACLF) score (range 0 to 100) is based on CLIF-C OF score incorporating additional variables of age and white blood cells count. Both scores were calculated using the CLIF research platform (www.clifresearch.com) to provide an estimate of the number of failed organs, the clinical severity, and the probability of death in the short and long-term follow up [13]. Model for end stage liver disease (MELD) score is based on total bilirubin, INR and serum creatinine and was calculated as 9.6 x log creatinine (mg/dL) + 3.8 x log bilirubin (mg/dL) + 11.2 x log INR + 6.43 [14]. United Kingdom model for end-stage liver disease (UKELD) is a variant of MELD that include serum sodium [15] and is currently used to allocate organs in United Kingdom's liver transplant list. The Sepsis-related Organ Failure Assessment (SOFA) provides an assessment of six organ systems: liver, renal, coagulation, cardiovascular, respiratory and central nervous system, the composite score ranging from 0 to 22 calculated on a 5 point grading scale (0 to 4) for each organ system [16]. Acute Physiology, Age, Chronic health Evaluation (APACHE) 2 (range 0-71) utilises the age of the patient, chronic health status, and a number of acute physiological variables including the worst value during the first 24 Hours of the heart rate, mean blood pressure, temperature, respiratory rate, PaO 2 , Alveolar-arterial gradient of Oxygen, haematocrit, white blood cells count, serum creatinine, presence of acute kidney failure, sodium, pH and Glasgow Coma Scale (GCS), [17]. APACHE 3 (range 0 to >299), additionally, also includes urine output, urea, glucose, total bilirubin, PaCO 2 and a different grading of GCS parameters [18]. ROC curve analysis was used to assess the performance of prognostic scores to predict 3-month mortality. The cut-off was identified by Youden index and its Hazard Ratio (HR) was identified by Cox regression analysis. The Area Under the Curve (AUC), sensitivity, specificity, along with positive predictive value (PPV) and negative predicting value (NPV) and p value were determined.

Developing a new score
The training cohort included NOLT patients admitted to RFH after 2001, the year that coincided with Norepinephrine as the preferred vasopressor of choice used in this setting. Multivariate Cox regression analysis was used to identify predictors of mortality between the first three best scores obtained from the previous analysis and the variables that were not part of them and resulted in significant association with mortality in univariate analysis. The factors showing a p<0.05 were used to develop the new score as follows: The ability to predict the mortality of the new model was assessed and compared with the existing scores by ROC analysis. The new score was validated in the cohort of non-transplanted patients from BJH.

Predictors of high risk of futility of LT
Patients from Beaujon hospital were used as explorative cohort. Futile LT was defined as occurrence of death within 48 hours of surgery in the context of development of MOF and/or irreversible brain damage but without major surgical complications (hepatic artery thrombosis, portal vein thrombosis, outflow obstruction, haemorrhagic shock) and/or primary graft non-function. The "non futile transplantation" included those who survived for at least 48 hrs following LT, and the deaths beyond this period were the direct result of transplant complications. Clinical and laboratory variables at the time of admission were analysed by Cox regression to identify the predictors of futility. The ability of the new score to predict futility was tested using ROC analysis.

Baseline patients' characteristics
152 patients admitted with acetaminophen related ALF were included in this study. As shown in Fig 1, 126 (82.9%) patients met the KCH criteria. Of them, 71 (56.3%) were listed for an emergency LT: 18 (25.4%) died on the waiting list, 9 (12.7%) improved without an LT and 44 (61.9%) were transplanted. (Fig 1). The mortality rate after 3 month from the hospitalization was 27.3%. Despite meeting the indication for an emergency LT, 18 (32.7%) were considered too sick to receive LT, 10 (18.2%) had psychiatric contraindication and 27 (49.1%) recovered spontaneously. Among patients who did not fulfil KCH criteria (n = 26/152, 17.1%), 6 (23.1%) were listed for LT: 3 were transplanted and survived at 3 months, while the other 3 improved and were removed from the waiting list. Of the 20 patients who did not fulfil KCH criteria and were not listed, only 1 died after 5 days due to MOF. As shown in Table 1, 98 patients were admitted to RFH and 54 to BJH. There were no differences between the two cohorts regarding the age, gender, 3-month mortality and fulfilment of KCH criteria during hospitalization (S1 Table).
However, more patients in the BJH cohort received LT than in RFH (51.9% vs 19.4%; p<0.001) and the median time to death from admission was significantly shorter (1 vs 8 days; p = 0.002). The BJH cohort also showed higher levels of serum ammonia (300 vs 98 umol/L; p<0.001), lactate (8.6 vs 4.8 mmol/L; p = 0.009) and norepinephrine requirement (15 vs 0 mcg/min; p = 0.004). No statistically significant differences were seen in the grade of HE and the use of organ support therapy in the first 24 hours of hospitalization.

Performance of existing scores
We analysed the 98 patients admitted in the intensive care unit of RFH. The mean age was 37.6±13.4 years and 66.3% were female. 64.3% presented with severe grades (3/4) of hepatic encephalopathy. An emergency LT was performed in 19/98 (19.3%) patients, all of them had met the King's College Hospital criteria. Of 79 NOLT patients, 13 were listed but did not receive a LT. Eight of them (61.5%) died on the waiting list and 5 (38.5%) spontaneously recovered. Of the 66 patients not listed, 16 (24.2%) did not fulfil KCH criteria; the remaining 50 patients met KCH criteria but LT was contraindicated in 15 patients (22.7%) who were too sick to receive an LT and 10 (15.2%) patients had psychiatric contraindication; 25 recovered spontaneously. As shown in Table 2, significantly higher number of LT patients needed mechanical ventilation (89.5% vs 62%; p = 0.022) than NOLT patients. The 3-month mortality rate was 37.8% and there was no difference between transplanted and non-transplanted patients (LT 36.8%, NOLT 39.2%, p = 0.847). All of death within 3 months among NOLT patients were due to MOF and only the 22.6% had culture-positive sepsis. In transplant patients 5 (71.4%) died due to sepsis, 1 (14.3%) due to MOF without sepsis and 1 (14.3%) committed suicide after 88 days from liver transplant. The survival time between the admission and death was significantly longer in the LT group (34 (5-92) vs 5.5 (0-21) days; p<0.001). The median MELD score and mean CLIF-C ACLF score were significantly higher in the LT cohort.
As shown in Table 3, in the NOLT cohort, all of the scores calculated at the time of admission had statistically significant ROC values. The KCH criteria had the lowest AUC (0.638) and the lowest PPV (49%) and prediction of poor outcome (the 3-month mortality) associated with a sensitivity of 83.9% and specificity of 43.7%. The highest sensitivity (100%) belonged to https://doi.org/10.1371/journal.pone.0188151.g001 APACHE 3 but it was associated with the lowest specificity (41.6%) that resulted in an AUC of 0.740. The best score was SOFA with an AUC of 0.799 followed by CLIF-C OF (0.793) and CLIF-C ACLF (0.762). CLIF-C OF showed a higher sensitivity than SOFA (93.5% vs 77.4%) and a lower specificity (58.3% vs 70.8%). There was no significant difference between these two AUCs (p = 0.849) and all of them were significantly higher when compared to KCH AUC (SOFA vs KCH p = 0.009; CLIF-C OF vs KCH p = 0.012). SOFA cut-off (11) was associated with a HR of 5.2 obtained by Cox univariate analysis (95% I.C. 2.13-12.73; p<0.001) and showed a PPV of 63.1% and a NPV of 82.9%. Patients with a CLIF-C OF score higher than the cut off (12) had a 38.9-fold increase in mortality risk (95% I.C. 1.66-907.57, p = 0.023). The CLIF-OF positive predictive value was 59.1% and the chance of a patient to survive with a CLIF-C OF score less than 12 (NPV) at the admission was 93.2%. In the LT group none of the scores calculated at the time of admission had statistically significant at ROC values in predicting the 3-month mortality from admission. Developing the new score Sixty-one consecutive patients admitted from 2001 at RFH comprised the exploratory cohort. The mortality rate after 3 months from hospital admission was 37.7%. Among the predictors of mortality identified in the univariate analysis (S2 Table), GCS, mean arterial pressure, pCO2, FiO2, platelets and INR and the use of vasopressor and mechanical ventilation were excluded from the multivariate since they were part of the best three scores obtained from the previous analysis. As reported in Table 4, the following factors were significant at univariate analysis and then included in multivariate Cox regression: body temperature, heart rate, alveolar-arterial gradient, dose of Norepinephrine, platelet count, albumin, aPTT, pH, potassium, base excess, SOFA, CLIF-C OF and CLIF-C ACLF. The variables significantly associated with 3-month mortality were CLIF-C OF (p = 0.014; B = 0.391; HR = 1.478; 95%IC 1.08-2.02) and the dose of norepinephrine required to maintain mean arterial pressure >70 mmHg (p = 0.012; B = 0.020; HR = 1.021; 95%CI 1.00-1.04).
The new score was calculated by multiplying the value of the predictors to their regression coefficient beta (B) and adding the results together Acute liver failure-Organ failure score (ALF-OFs) = (CLIF-C OF x 0.391)+(Norepinephrine mcg/min x 0.020). ALF-OFs was tested in the exploratory cohort and ranged between 5.2 and 11.3. In ROC analysis its AUC (0.890) was significantly higher when compared with CLIF-C OF (p = 0.044) and KCH (p<0.0001). The best cut-off was 5.58 characterized by a sensitivity of 82.6%, a negative predictive value of 89.5% and the highest specificity (89.5%) and PPV (82.6%) among the five tested scores (Fig 2A). As showed in Kaplan-Meier curve (Fig 2B), there was a significant difference in the 3-month mortality when dividing the patients according to the ALF-OFs score cut-off of 5.58 (>5.58 = 67.9% vs <5.58 = 12.1%; p<0.001). The validation cohort was composed of 26 non-transplanted patients admitted at BJH. 13/26 (50%) died within 3 months from the hospitalisation due to MOF. As showed in Fig 2B, ALF-OFs achieved an AUC of 0.988 with a sensitivity of 100% and a specificity of 92.3% at ROC analysis. Its curve was also significantly higher (p = 0.0013) when compared to KCH's AUC (0.731).

Risk of futile LT
We analysed 28 patients from BJH who received an emergency LT for acetaminophen-related ALF. Of them, 4 (14.3%) died within 3 days from the theatre due to a MOF and without any surgical complication. We considered them in the group that was at high risk of futility. We included in the non-futile group 24 patients that were still alive after 1 month from the liver transplant. All of the patients that died early required a multiorgan support treatment since hospital admission. They showed higher heart rate (114 vs 106bpm; p = 0.022), lower PaO 2 / FiO 2 rates (14.9 vs 69.1kPa; p = 0.043) and required significant higher doses of Norepinephrine (68.1 vs 0 mcg/min; p = 0.035) ( Table 5). ALF-OFs was significantly higher in those that died early on univariate Cox regression (p = 0.021, Hazard Ratio 2.955, 95% CI 1.18-7.40). At ROC analysis (Fig 3), the new score achieved an AUC of 0.917 (p<0.0001). The best cut-off was 6.5 with a 100% of sensitivity and 79.2% of specificity (PPV = 44.5%; NPV = 100%).

Discussion
KCH criteria [5] have been extensively used to identify patients who need an emergency LT. However, recent meta-analyses showed that KCH had poor sensitivity in predicting both the LT-free mortality [6,19] and the outcome after liver transplantation [8,20]. Several attempts have been made to create new prognostic models [8,[21][22][23]. However, most of them have underestimated the role of multiple-organ failure in the context of ALF [24]. The drastic organ shortage dictates that candidates for LT should be selected taking into account both the risk of death with medical management alone and the probability of survival after transplantation. This study aimed to develop a new prognostic score that could predict the 3-month mortality in patients with acetaminophen induced ALF and at the same time identify the patients in whom an emergency liver transplant at high risk of futility. Two cohorts from distinct intensive care units comprised our study population. They were similarly matched for age, sex, HE grade at presentation and need for organ support. The overall 3-month mortality rate in NOLT and LT cohort were similar between the two centres and in line with the currently available literature [25][26][27]. However, the percentage of patients who fulfilled poor prognostic   criteria and were listed for LT that was higher in BJH group. It might be due to a different approach between the two centers. These potential differences may be due to the following reasons. First, in RFH group the predominant reasons for no-listing despite fulfilling KCH criteria were non-liver contraindications for liver transplantation. Second, in London the approach was to stabilize the patients before the surgery. Therefore, patients who improved with a conservative therapy were removed from the waiting list. Moreover, patients that deteriorated showing evidence of multiorgan failure and a poor response to treatment were not listed or, if listed, removed from the waiting list because the benefit of liver transplantation was considered too low. This policy could also explain the absence of futile liver transplantion in the Royal Free group. Third, the high proportion of patients considered too sick to be listed might also be due to a late referral to RFH, London. Indeed, most of them had sepsis at the admission. KCH criteria were used to select LT candidates in both cohorts. However, only one third of patients fulfilling the KCH were transplanted. Therefore, NOLT group was generated including four different categories: patients that did not fulfil KCH, those who died while on waiting list, those who improved spontaneously and those who were not listed for clinical or psychiatric reasons. This cohort was characterized by a wide range of ALF severity and it could be considered a reliable population for the creation of a new prognostic model. As the first step, we tested the performance of the main prognostic scores used in hepatology and intensive care units to predict the 3-month mortality in patients receiving LT or not. Among NOLT group, the scores that explore the severity of MOF, such as SOFA, CLIF-OF and CLIF-ACLF, showed the best AUCs. These results highlight the important prognostic role of hemodynamic dysfunction even in the early stage of ALF [24,28]. On the other hand, none of the scores calculated at admission was able to predict the 3-month mortality in LT patients, confirming that post-LT outcome is strongly affected by multiple factors including immunosuppression and graft characteristics [29]. The second step in our strategy was to look for independent predictors of 3-month transplant-free mortality. Since survival in patients with ALF has significantly improved after year 2000 due to the advances in critical medical care [29][30][31], we decided to analyse only patients admitted after this time point. The multivariate analysis included the scores with the best performance (SOFA, CLIF-OF and CLIF-ACLF) along with the variables tested significant in the univariate analysis and that were not present in these scores. CLIF-C OF score and the level of requirements for norepinephrine were the only significant predictors and therefore were used to build the new score. ALF-OFs score resulted a modified version of CLIF-C OF where a greater importance is given to the cardiovascular dysfunction that has been shown to significantly affect ALF patients' outcome [24]. This could explain why our new model performs better than the existing score with a good balance between sensitivity and specificity both in the exploratory and validation cohorts. The last step of our study consisted of identifying the predictors of patients at high risk of early mortality after liver transplantation. Previously published studies have shown that survival after liver transplantation for ALF is still lower compared to other aetiologies, indicating that a better understanding of poor prognostic factors is mandatory to optimize organ allocation. No consensus exists on the definition of criteria that defines futility of LT [30]. The three large studies, that explored the outcome after an emergency LT, did not analyse the different aetiology (APAP-OD vs. non APAP-OD) separately and they did not discriminate patients according to cause of death [10,29,32]. We decided to focus on the highest risk group defining a high risk of futile LT as the occurrence of death within 48 hours due to MOF or irreversible brain damage that were not related to graft-function, immunosuppression and surgical complication. This definition allowed us to identify those patients in whom an emergency LT did not provide a clinical benefit. Our findings showed that early deaths after LT were characterized by a greater pre-transplant circulatory dysfunction, as suggested by the higher requirement of vasopressor in this group. ALF-OFs showed a good performance also in predicting the futility characterized by a high sensitivity (100%) associated with an acceptable specificity (79.2%). Moreover ALF-OFs allowed us to stratify the mortality risk in APAP-OD-ALF. Given the relatively small number of patients in this part of the study, we suggest caution in applying these criteria without further validation to clinical practice. In conclusion, as shown in Fig 4, we identified two different cut-offs that allowed us to further classify the patients into three categories; patients who are likely to survive without a LT (ALF-OFs<4.5); patients with a high risk to die without a LT (ALF-OFs 4.5-8.5); and those where a LT is at high risk of being futile (ALF-OFs>8.5). We acknowledge that the numbers of patients are limited and larger studies are needed to further investigate this topic. However, this study points to the role of multiple organs in defining the outcome of ALF patients and describes a new prognostic score based on pre-LT variables to define the need for LT and early post LT mortality.
Supporting information S1