Correction: The Singapore Liver Cancer Recurrence (SLICER) Score for Relapse Prediction in Patients with Surgically Resected Hepatocellular Carcinoma

There is an error in the order of authors. The correct order is: Soo Fan Ang1*, Elizabeth Shu-Hui Ng1, Huihua Li2,3, Yu-Han Ong1, Su Pin Choo1, Joanne Ngeow1, Han Chong Toh1, Kiat Hon Lim4, Hao Yun Yap5, Chee Kiat Tan6, London Lucien Peng Jin Ooi7, Peng Chung Cheow7, Alexander Yaw Fui Chung7, Pierce Kah Hoe Chow8,9, Kian Fong Foo1, Min-Han Tan1* 
 
1 Division of Medical Oncology, National Cancer Centre Singapore, Singapore, Republic of Singapore, 2 Health Services Research, Singapore General Hospital, Singapore, Republic of Singapore, 3 Centre for Quantitative Medicine, Duke-National University of Singapore Graduate Medical School, Singapore, Republic of Singapore, 4 Department of Pathology, Singapore General Hospital, Singapore, Republic of Singapore, 5 Department of General Surgery, Singapore General Hospital, Singapore, Republic of Singapore, 6 Department of Gastroenterology and Hepatology, Singapore General Hospital, Singapore, Republic of Singapore, 7 Department of Hepatobiliary and Transplant Surgery, Singapore General Hospital, Singapore, Republic of Singapore, 8 Division of Surgical Oncology, National Cancer Centre Singapore, Singapore, Republic of Singapore, 9 Office of Clinical Sciences, Duke-National University of Singapore Graduate Medical School, Singapore, Republic of Singapore.


Introduction
Hepatocellular carcinoma is often associated with a poor prognosis and is responsible for a disproportionately high global burden of morbidity and mortality. Its incidence is increasing in several developed countries, particularly in Asia as a result of a cohort effect related to infection with hepatitis B and C viruses [1]. To date, surgical resection remains the gold standard treatment in patients with adequate residual liver function, and liver transplant offers the best long term outcomes for patients with impaired liver function secondary to liver cirrhosis. Ablative modalities such as radiofrequency ablation or trans-arterial chemo-embolization are frequently employed for palliative treatment or as a bridge to liver transplant. Despite successful surgical resection and the use of antiviral drugs in the setting of hepatitis-induced liver cirrhosis, the risk of relapse is still extremely high with tumor recurrence developing in up to 70% of cases at 5 years [2].
There have been several scoring systems developed for classification and prognostication of HCC, and these include the American Joint Committee on Cancer staging system 7 th edition (AJCC7), Okuda score, Barcelona Clinic Liver Cancer (BCLC), Cancer of the Liver Italian Program (CLIP), Chinese University Prognostic Index (CUPI) and Japan Integrated Staging Score (JIS score) [3][4][5][6][7][8][9]. These are predominantly derived from patients with metastatic and locally advanced disease, often with impaired liver function, and have not been validated for use in prediction of relapse after surgical resection. These scoring systems only serve to classify patients into various groups with varying outcomes, but do not predict individualized outcomes. One nomogram based on a smaller dataset in the USA has been proposed to predict disease free survival, and another has been proposed to predict pulmonary metastases, but to date both have not been externally validated [10,11].
From a clinical perspective, there is a need for an accurate model for predicting individualized probabilities of HCC recurrence after curative liver resection. This would guide patient counseling and effective scheduling of clinical surveillance, which is important as early detection of recurrence could be amenable to further curative surgical resection. The model would also help in stratifying patients who may benefit from adjuvant treatment, rank potential liver transplant candidates and serve as a basis for patient selection in clinical trials.
In this study, we have constructed a new postoperative nomogram, the Singapore Liver Cancer Recurrence (SLICER) Score, to predict the probability of freedom from recurrence in patients who have undergone curative surgical resection for HCC. We also demonstrate that it performs better than several major HCC staging systems in use today in predicting probability of freedom from recurrence.

Ethics Statement
Institutional review board approval from the Singapore Health Services was obtained for this study. All patient records and information was anonymized and de-identified prior to analysis.

Patients
Patients who underwent primary curative resection for HCC were identified through the hospital database and their medical records were reviewed. We limited our dataset to Singaporean patients who underwent surgery between 1992 and 2007, both to reduce sampling and followup bias, as well as to allow for a sufficient duration of post-resection follow-up data to be obtained. All patients underwent a chest x-ray, and either a liver computed-tomography (CT) scan or magnetic resonance imaging (MRI) of the liver prior to surgery. Clinical, radiological and pathological data of these patients were extracted for analysis. The pathological specimens and slides were reviewed by a pathologist specialized in hepatobiliary pathology and tumor characteristics, including but not limited to tumor size, encapsulation, presence or absence of cirrhosis in non-cancerous tissues, resection margin, grade and vascular invasion, were reported. CLIP, CUPI, BCLC, Okuda, Child-Pugh scores and AJCC7 were determined from available data.
All patients were followed up post-operatively according to standard department practices at maximum intervals of 6 months. Clinical surveillance consisted of regular clinical evaluations, serum alpha-fetoprotein (AFP) levels and hepatic imaging in the form of ultrasonography, CT scan or MRI as deemed appropriate by the surgeon. All imaging done was reviewed and reported by radiologists specialized in hepatobiliary imaging. A relapse was diagnosed based on new intra-hepatic or extra-hepatic lesions characteristic of HCC seen on imaging.

Statistical analysis
The clinical endpoint was freedom from recurrence (FFR). This was defined as the time from date of surgery to date of first relapse detected on imaging, or to the last follow up date for censored cases. A Kaplan-Meier survival curve was used to estimate FFR and univariable Cox regression was carried out to evaluate the effects of various clinicopathologic variables on FFR. Clustering of variables was conducted to reduce the number of potential prognostic factors by evaluating the similarity of these factors [12]. Subsequently, reduced model selection was performed using a backward stepdown by applying the Akaike information criterion [13]. Proportional hazards assumptions were verified systematically for all proposed models. The final multivariable Cox regression coefficients were used to construct the SLICER nomogram. Internal validation was performed to evaluate the ability of SLICER nomogram to predict FFR of individual patients with 200 sets of bootstrap samples as it replicated the development and validation cycle 200 times and used a full or nearly full version of the dataset for each cycle [12]. Calibration plots were drawn to explore the performance characteristics of the nomogram at 3 and 5 years post-resection [12].
Likelihood ratio testing of nested models were performed to compare SLICER to the other prognostic models including CLIP, CUPI, BCLC, OKUDA, AJCC7 and Memorial Sloan-Kettering Cancer Centre (MSKCC) nomogram on a pairwise basis [12]. An adequacy index was used to quantify the percentage of the variation explained by a subset of the individual predictors compared with the information contained in the full set of predictors by means of log-likelihood [12]. Harrell's c-index was calculated to evaluate the concordance between predicted and observed responses of individual subjects separately [12]. Decision curve analyses, plots of net benefit against threshold probability, were carried out to evaluate these predictive models by examining the theoretical relation between the threshold probability of developing an event and the relative value of false-positive and false negative results as described by Vickers et al [14].
All statistical analyses were done using R 3.0.2 (http://www.R-project.org) and STATA 11 (STATA Corporation, College Station, TX USA), and all tests were two-sided with a significance level of 0.05.

Patient characteristics
Between 1992 and 2007, 544 consecutive patients who underwent first-line curative hepatic resection for HCC in Singapore General Hospital were recruited. We excluded foreign patients and patients with missing data (n = 139), leaving 405 patients to form the cohort for our analysis. Their baseline demographic and clinical characteristics are shown in Table 1. The median age of the cohort was 64 years (range 30-88 years), of which 81.7% were male. The cohort consisted predominantly of patients of Chinese ethnicity (92.6%). 62.5% of our patients had either a positive hepatitis B surface antigen, chronic hepatitis C infection, or both. Most patients (89.9%) had biochemical results consistent with Child-Pugh class A status (192 without pathologic cirrhosis), early stage disease (60% with AJCC7 stage I). 265 patients (65.4%) were asymptomatic at diagnosis.

Prognosis
The median follow-up was 25.8 months (range 0.03-173.3 months). A total of222 patients relapsed, and there were 171 deaths. The median overall survival of the cohort was 55.9 months and median FFR was 25.2 months.

Nomogram construction
Univariable analysis showed that Child-Pugh class status, pre-surgery AFP level, non-neoplastic liver cirrhosis, multifocality, vascular invasion, tumor size, margin distance, symptoms, ECOG (Eastern Cooperative Oncology Group) performance status, histological grade and AJCC7 stage significantly affected FFR (Table 2). Among these, ECOG status and AJCC7 stage were excluded from the multivariable analysis as Hoeffding' D statistics showed that ECOG status and symptoms, vascular invasion and AJCC7 stage were highly correlated ( Fig. 1). Finally, 8 of them were found statistically significant in multivariable analysis and selected to construct the nomogram (Table 3, Fig. 2). These include the presence or absence of cirrhosis in the non-neoplastic liver, tumor multifocality, pre-surgery AFP, Child-Pugh class status, vascular invasion, tumor size, margin distance and the presence or absence of symptoms at diagnosis. Bootstrapping was then performed to determine the calibration accuracy for 3-year and 5-year FFR estimates from the final Cox model. Calibration curves were plotted showing that the maximum deviation from ideal were 5.6% and 3.9% respectively (Fig. 3).

Comparison of SLICER to alternative staging models as predictor of FFR
The SLICER score demonstrated superior predictive capabilities relative to the other models with a hazard ratio of 1.05 (95%CI 1.04-1.06) and concordance index of 0.69 for prediction of FFR (Table 4). On decision curve analysis [13], compared to other models, our nomogram showed equivalent net benefit between 0-40% threshold probability, but improved performance beyond 40% threshold. This represents superior estimation of decision outcomes at higher threshold probability levels (Fig. 4). The adequacy index of SLICER was also higher when compared to the other models individually. In terms of likelihood analysis, its inclusion in a full model resulted in highly statistically significant improvements when tested with logistic regression analysis against the CLIP (p<0.0001), CUPI (p<0.0001), BCLC (p<0.0001), OKUDA (p<0.0001), AJCC7 (p<0.0001) and MSKCC (p<0.0001) (Fig. 5).

Discussion
HCC is one of the most common malignancies worldwide, especially in Asia in association with a high incidence of Hepatitis B and C infection. Surgical resection offers the best prognosis for long term survival, but little is known about prevention of relapse post-resection and rates of relapse vary over a wide range in published studies. In our centre, we include patients who have an ECOG score of 1, who would otherwise be excluded from surgical resection based on BCLC staging recommendations. This accounts for the high percentage (39.5%) of BCLC stage C patients included in our study. Several scoring systems for estimation of prognostic outcomes in HCC have been developed to risk stratify patients [15][16][17], and these systems primarily classify patients in to various risk groups. Nomograms can give each factor a relative weight in prognostication of the outcome, and permit more refined risk estimation for each individual. However, they are challenging to create in a very heterogenous disease such as HCC as many factors need to be considered. In our study, we examined multiple pre-operative and post-operative prognostic factors, and developed a nomogram to quantify expected individualized recurrence outcomes for HCC patients treated by first-line surgical resection. We ensured that these variables are common parameters that are routinely done and easily obtainable for all HCC patients, across all institutions. The SLICER score is meant for use in post-operative patients, and is not meant for preoperative counseling of the utility of surgery.
To date, one nomogram has been developed for risk estimation of liver cancer recurrence and survival post-hepatectomy at the MSKCC [10]. This nomogram was generated from a small series of 184 patients, and had evident divergence between observed and expected outcomes during calibration, possibly due to the low numbers involved. This nomogram performed less well in our dataset, possibly due to the different patient profile of HCC in the Asian population. In our nomogram, patient age was not a defining variable and the presence of symptoms at diagnosis indicated a higher score. This probably reflects a larger tumor or a later stage tumor at diagnosis, thus translating to an increased risk of relapse.
Risk factors for postoperative recurrence after resection of HCC are related to tumor, host, and surgical factors. Pathologic factors indicative of tumor invasiveness such as vascular invasion, multifocality, large tumor size, and advanced pathological TNM (Tumor, Node, Metastasis) stage are well-established risk factors for recurrence [18]. Microscopic venous invasion as  well as macroscopic portal vein involvement are both major risk factors as metastasis via portal venous system is an important mechanism for intrahepatic recurrences [19,20]. The effects of tumor encapsulation and histologic differentiation exert on the risk of recurrence are less conclusive. Active inflammatory activity of the non-neoplastic liver has been associated with a higher proliferative activity and is an independent risk factor for intrahepatic recurrence. Subclinical presentation of HCC is however an independent favorable prognostic factor for disease-free survival [21]. A few studies had identified perioperative blood transfusion as a risk factor for HCC recurrence. Most studies found that the extent of resection and whether anatomical or non-anatomical, had no significant influence on the risk of recurrence. The effect and significance of underlying cirrhosis in the non-neoplastic liver, a wide resection margin to ensure histologic clearance and administration of neo-adjuvant or adjuvant therapy, on the risk of recurrence after resection of HCC, has remained controversial [18]. The results shown in our analysis were consistent with these previous findings.
There has been literature to suggest that early and late recurrence of HCC are thought to be different-early recurrence thought to be a result of metastasis from the same tumor and late recurrence thought to be due to de-novo tumors occurring on a background of field change. The variables we have identified account for both the tumor characteristics (eg. vascular invasion, multifocality, tumor size) and the possible effect of field change (liver cirrhosis, Child-Pugh score), thus accounting for both early and late recurrences of HCC.
More than half our study population (55.6%) are known to be Hepatitis B carriers, reflecting a slightly unique patient profile of HCC patients in Asia. Hepatitis B viral load is known to be an important predictor of HCC recurrence, with most patients being treated with long-term anti-viral therapy as it is thought to reduce HCC recurrence. [22] Unfortunately, in the early part of our study, Hepatitis B viral load was not a routine investigation done for patients and anti-viral therapy was not readily available, thus this variable was not included in our nomogram.
In recent years, molecular research has identified various biomarkers as predictive and prognostic markers for HCC metastatic recurrence and clinical outcomes, including tumor-associated antigens (such as AFP, MAGEs, GPC3, and CK19), molecular factors associated with HCC invasion and metastasis (such as E-cadherin, catenins, ICAM-1, laminin-5), and angiogenesis regulators [23,24]. In particular, elevation of the serum fucosylated fraction of alpha-fetoprotein (AFP-L3) level before treatment is a predictor of HCC recurrence, and sustained elevation of the AFP-L3 level after treatment is an indicator of HCC recurrence [25]. A high pre-operative level of EpCAM-positive circulating tumour cells and some gene expression signatures have also been found to be a predictor for recurrence [26,27]. However the clinical applications and availability of these biomarkers is limited and more extensive studies are needed for validation. The Singapore Liver Cancer Recurrence (SLICER) Score It is useful to note that molecular studies have shown that 60-70% of recurrences are due to true metastasis that results from HCC dissemination before resection [28] and occurs mainly within the first few years after resection [29]. The SLICER score may be predicting a composite of these outcomes. Potential interventions for high-risk patients identified by the nomogram may include systemic and hepatic-artery chemotherapy or chemoembolization; however, these have not been shown to improve overall or disease-free survival after resection of HCC [30]. Early deceased-donor or living-donor liver transplantation is also a consideration. The emergence of targeted therapy such as sorafenib has expanded the scope of therapy in advanced HCC [31,32]. Consequently, there are several ongoing adjuvant clinical trials evaluating agents such as sorafenib and gefitinib for HCC patients after potentially curative treatment [11,33], settings where the SLICER score may be particularly appropriate. While survival remains the main endpoint to measure effectiveness in phase 3 studies, freedom from recurrence is a useful and clinically appropriate endpoint in the adjuvant setting after curative treatment for HCC [34], particularly since repeat resection remains a viable option. A validated nomogram or model therefore is important to assist in patient selection and future clinical trial design and biomarker research.
Our results demonstrate that SLICER outperforms other existing established HCC prognostic models in estimation of FFR, and even more importantly, is well calibrated. Certainly it is true that CLIP, CUPI, BCLC, Okuda scores and AJCC 7 th staging systems were developed primarily to predict patient survival rather than relapse. In addition, the data used to develop Repeat this process for other six variables and sum up the points achieved for each variable. The sum of these numbers is located on the Total Points axis, and a line is drawn downward to the survival axes to determine the likelihood of 3-or 5-year FFR. For example, a patient who has a 3 cm HCC with multifocality, liver cirrhosis, Child-Pugh A, minor vascular invasion, resection margin 5 mm, pre-surgery AFP 450ng/mL and he was asymptomatic at presentation, total points scored is 48. 3-and 5-years FFR is 16 and 8% respectively. The Singapore Liver Cancer Recurrence (SLICER) Score these models derive mainly from patients with advanced HCC who are not candidates for curative treatment and who generally have poorer liver function. There is thus a clear need for the The Singapore Liver Cancer Recurrence (SLICER) Score nomogram we have developed from the largest series to date reported, given the difference in patient profiles between patients with localized and patients with advanced disease. Even more importantly, in addition to discriminatory accuracy, our model was able to provide considerably superior estimation of clinical net benefit at higher threshold probability levels, and comparable estimation of net benefit at lower threshold probability levels, compared to other staging systems. It is important to note that the x-axis of the decision curve analysis diagram is not a direct measurement of model accuracy or performance. It represents a general presentation of estimating decision outcomes (presented as 'net benefit') across a range of different risk level thresholds, with two extremes 'all' or 'none' representing two separate extreme assumptions ('all' will relapse and 'none' will relapse). It may be seen from the Fig. 4 (top left image) for example that the SLICER and CLIP models showed equivalent net benefit between 0%-40% decision threshold probability, but SLICER showed considerably improved performance over CLIP beyond 40% decision threshold.
Our study indicated that CUPI was least precise for predicting FFR among the 6 staging models tested, with a c-index of only 0.54. CLIP and CUPI staging models have been reported as the most informative regarding survival outcome in advanced HCC [35]. The BCLC classifications stratify HCC into different stages and defines standard of care for each tumor stage. BCLC has been widely used as standard classification for trial design and clinical management of patients with HCC [36]. In our local patient cohort, BCLC does outperform CLIP or CUPI The Singapore Liver Cancer Recurrence (SLICER) Score Dashed lines indicate the net benefit of SLICER in each of the curves across a range of threshold probabilities. The horizontal solid black line represents the assumptions that no patients will experience the event, and the solid gray line represents the assumption that all patients will relapse. On decision curve analysis, SLICER showed superior net benefit compared to other models across a range of threshold probabilities. doi:10.1371/journal.pone.0118658.g004 The Singapore Liver Cancer Recurrence (SLICER) Score with regard to predicting ability for FFR, but is in turn outperformed by SLICER. Likelihood analysis also showed that inclusion of SLICER in a full model resulted in highly statistically significant improvements when tested with logistic regression analysis against the other models. Limitations to our study include its retrospective nature, and that it is based on a single-institution experience. As our centre sees approximately 70% to 80% of all surgically resectable patients nation-wide, this may reduce the concern of bias arising from data from a single institution. External validation at other centres will be required. We did not manage to include some other variables known to affect recurrence such as iatrogenic intra-operative tumour rupture [37], or histopathological subtypes of HCC. Fibrolamellar HCC occurs in a distinctly different population of patients than common HCC, and is known to have relatively indolent tumour biology with a better prognosis [38].
Ideally, SLICER should also be paired with a better surveillance model to determine if highrisk patients as identified by SLICER truly benefit from closer surveillance. To date, there are no randomized trials of surveillance strategies in post-operative HCC to determine the benefit of risk-adjusted strategies and SLICER could serve as an appropriate prognostic model for such trials.
To our knowledge, the SLICER score here is developed on the largest reported series of post-surgical patients to date. We have developed a model to predict individualized probabilities of relapse following complete resection of localized hepatocellular carcinoma, in a large, near-national consecutive series of patients, which demonstrates excellent calibration and performance. This provides a foundation for individualized patient counseling and management, biomarker development, and trial design for adjuvant trials in HCC.