Figures
Abstract
Objective
At present, only the number of metastatic lymph nodes (LNs+) is used for the pN category of AJCC TNM system for colon cancer. Recently, the ratio of metastatic to examined lymph nodes (LNR) has been reported to represent powerful independent predictive capacity in colon cancer. We sought to propose a novel category (nLN) which intergrades LNR and LNs+ into the AJCC staging system for colon cancer.
Design
34476 patients from the National Cancer Institute's Surveillance, Epidemiology, and End Results (SEER) dataset with stage III colon cancer were reviewed. Harrell's C statistic was used to evaluate the predictive capacity. The Cox proportional hazards model was used to construct a novel category.
Results
The LNR category had more predictive capacity than the pN category in whole groups of patients (Harrell's C index: 0.6194 vs 0.6113, p = 0.003). Subgroup analysis showed that the LNR category was not better than pN category in predictive capacity if the number of lymph nodes examined was more than 13. We also found that there was significant survival heterogeneity among different pN categories at the same LNR category (P<0.001). The Harrell's C index for our nLN category which intergrades LNR and LNs+ was 0.6228, which was significant higher than that of the pN category (Harrell's C index: 0.6113, P<0.001) or LNR category (Harrell's C index: 0.6194, P = 0.005), respectively.
Citation: Gao P, Song Y-x, Wang Z-n, Xu Y-y, Tong L-l, Zhu J-l, et al. (2012) Integrated Ratio of Metastatic to Examined Lymph Nodes and Number of Metastatic Lymph Nodes into the AJCC Staging System for Colon Cancer. PLoS ONE 7(4): e35021. https://doi.org/10.1371/journal.pone.0035021
Editor: Ajay Goel, Baylor University Medical Center, United States of America
Received: November 14, 2011; Accepted: March 8, 2012; Published: April 18, 2012
Copyright: © 2012 Gao et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This work was supported by National Science Foundation of China (no. 30972879 and no. 81172370), Specialized Research Fund for the Doctoral Program of Higher Education (no. 200801590006), Natural Science Foundation of Liaoning Province (no. 20092129), the Program of Scientific and Technological Department of Liaoning Province (no. 2010225032) and the Program of Education Department of Liaoning Province (L2011137). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Colon cancer is one of the most common malignancies [1]. The International American Joint Committee on Cancer (AJCC) TNM staging system is currently regarded as the strongest prognostic parameter for patients with colon cancer [2]. Lymph node metastasis is one of the most important prognostic factors. Determination of the optimal approach to quantifying lymph node status in colon cancer will ensure accurate patient staging, allowing appropriate adjuvant treatment planning and calculation of long-term prognosis.
At present, only the number of metastatic lymph nodes (LNs+) is used for the pN category of AJCC TNM system for colon cancer. This has been criticized as an oversimplification because the number of metastatic lymph nodes is influenced by the total number of examined lymph nodes (eLNs) and may increase the probability of stage migration [3], [4]. As we know, the eLNs pathologically has been demonstrated to affect both staging accuracy and oncological outcomes in node-positive patients [5]. The optimal eLNs for reliable prognostic stratification is less clear until now. According to the guidelines from the AJCC, a minimum of 10–14 lymph nodes must be examined and histopathologically assessed in the tumor specimen to adequately evaluate lymph node status [6]. The college of American Pathologists recommends a minimum of 12 lymph nodes to be examined for colon cancer [7]. Some researchers also proposed that the resection of at least 13, 14 or 15 nodes was associated with prolonged survival in colon cancer for the categories examined [8]–[10]. Unfortunately, surgeons and pathologists do not generally succeed in meeting minimal nodal staging. For those cases without an adequate number of retrieved lymph nodes, the pN category may be not accurate enough.
Over the past few years, the ratio of metastatic to examined lymph nodes (LNR) has been studied widely. Nearly all researchers demonstrated that the LNR is an independent prognostic factor that is highly related to the survival of patients with colon cancer and it has been recommend that the LNR should be applied in prognostic assessment [11]–[18]. However, it is still unclear whether the LNR has more prognostic validity than the AJCC pN category [18], [19].
This study is based on a dataset supported by the Surveillance, Epidemiology, and End Results (SEER) cancer registry with 34476 cases that suffered colon cancer. We found that the LNR category had more predictive capacity than the pN category in the whole groups of patients. However, if the eLNs was more than 13, the LNR category was not better than pN category in predictive capacity. Additionally, there was significant survival heterogeneity among different pN categories at the same LNR category. Finally, we proposed a new category approach that intergraded LNR and LNs+ into the AJCC staging system for colon cancer.
Materials and Methods
Data
The dataset we used is the National Cancer Institute's Surveillance, Epidemiology, and End Results (SEER) dataset, 1973–2007. SEER collects data on cancer cases from various locations and sources throughout the United States. Data collection began in 1973 with a limited amount of registries and continues to expand to include even more areas and demographics today. The number of records in the SEER research dataset is up to 6127828 including 5564451 malignant cases. Among these patients, more than 500000 patients suffered from colorectal cancer. Patients with stage III colon cancer diagnosed from 1992 through 2003 were selected for analysis. The primary study endpoint was cancer-specific survival.
Patients were excluded from this study if they had: 1) a prior non-colon cancer or colon cancer other than adenocarcinoma or mucinous adenocarcinomas 2) underwent preoperative radiation, because it was reported that the total number of retrieved lymph nodes may decrease after preoperative chemoradiation [20]; 3) incomplete pathological data entries; or 4) died during the immediate postoperative period (within one month).
After using these exclusion strategies, a dataset consisting of 34476 records was constructed and the following data were recorded: age, gender, race, depth of invasion (determined by SEER's “extent of disease”), histologic grade, number of lymph nodes retrieved, and number of metastatic lymph nodes. Then, the LNR was defined as the ratio of LNs+ divided by the eLNs. To avoid some biases such as the complex category may be over optimized in the comparison of predictive capacity between different categories, models constructed by the categories were found in a training set of data, and then their predictive capacity determined in a test set of data, independent of the training set [21]. Therefore, of the 34476 cases, half were randomly selected for training and the remaining 17238 were used for testing.
Statistical Analysis
Continuous data were presented as the mean ± standard deviation (SD). Cancer-specific survival was analyzed by Kaplan-Meier survival curves, and comparisons were made by the log-rank test. Multivariate analysis was performed using Cox proportional hazards model.
We evaluated the predictive capacity of categories by considering measures of discrimination. Discrimination refers to the ability to distinguish between high-risk and low-risk patients, and was quantified using the Harrell's C statistic, Nagelkerke R2, Bayesian Information Criterion (BIC), and time dependent cumulative area under the curve (AUC) [22]–[26]. A model with perfect predictive capacity (sensitivity and specificity of 100%) would have a Harrell's C index of 1.00; a category with higher Harrell's C index was considered more accurate in predictive capacity. The Nagelkerke R2 index was also used to score the different categories. R2 represents the proportion of variation explained by covariates in regression models. R2 is close to 1 for a perfectly predictive model, and close to 0 for a category that does not discriminate between short and long survival times. The BIC was used to assess the overall prognostic performance of different classification systems via bootstrap-resampling analysis. A smaller BIC value indicates a more desirable model for predicting outcome. The AUC was a common tool for the purpose of assessing the predictive power of a continuous variable for a binary outcome and cumulative AUC which was an extension of it to censored survival data was used to evaluate the accuracy of categories in survival prediction at different time.
The cut-off values for sub-groups of LNR were determined using Harrell's C statistic calculated from the training dataset [8], [22]. To study whether the predictive capacity of the LNR category is better than pN category with set standards for the minimal eLNs, a series of tests were performed. There were 20 tests run using a standard for the minimal eLNs from 2 to 21. In each test, a Harrell's C statistic was determined to test the predictive capacity of LNR categories and pN categories.
Comparison of the survival rate among different pN categories stratified by LNR categories was run to analyze the heterogeneity. A log-rank test was run to compare the survival rate among different pN categories in each LNR category.
The novel category (nLN) which combines the pN category with the LNR category is based on the hazard ratio calculated by Cox proportional hazards model. The formula of Cox proportional hazards model is: , where
…
are a collection of predictor variables, LNs+ and LNR in this study,
…
are regression coefficients determined by a least squares approach, and the
is called the hazard ratio. Moreover, we grouped the calculated hazard ratio four risk levels and formed our nLN category and the optimal cut-off values for the nLN category were also determined using Harrell's C statistic calculated from the training dataset. And then, we compared the predictive capacities of this nLN category with single LNR category and single pN category. Moreover, to test whether the nLN category will have more predictive values irrespective of the eLNs, comparison of the survival rate between patients with <12 eLNs and ≥12 eLNs stratified by all three categories was run.
All the statistical analyses and graphics were performed with the PASW Statistics 18.0 software (SPSS, Inc., Somers, NY, USA), SigmaPlot 12.0 (Systat Software Inc), R version 2.14.0 (The R Foundation for Statistical Computing), Splus 8.0 (Insightful Corporation, Seattle, WA, USA) and STATA MP ver.10 (StataCorp LP, College Station, TX) statistical software. For all analysis, P<0.05 was considered significant.
Results
According to the 7th edition of the UICC/AJCC TNM staging system, based on the number of positive lymph nodes, patients with different pN categories were divided into: N1a, 34.3% (11826/34476); N1b, 33.8% (11665/34476); N2a, 19.6% (6747/34476); and N2b, 12.3% (4238/34476). Survival differences among the groups were statistically significant (P<0.001; Table 1).
Based on optimal cut-off values determined using Harrell's C statistic respectively, patients were divided into the following LNR subgroups: LNR1 = an LNR<0.13; LNR2 = an LNR between 0.13 and 0.24; LNR3 = an LNR between 0.24 and 0.51; and LNR4 = an LNR>0.51. The 5-year survival rate decreased significantly with increasing LNR categories (P<0.001; Table 1).
Moreover, in univariate analysis, age, race, histologic grade and pT categories were also identified as significantly correlated with prognosis (Table 1). In the multivariate analysis, all clinicopathological factors that were significantly correlated with prognosis in univariate analysis were considered. Age, race, histologic grade, pT categories, pN categories, and LNR categories were confirmed to be independent prognostic factors (Table 2). Using Harrell's C statistic to test the predictive capacity of the category in all patients, the LNR categories was significantly better than the pN categories (Harrell's C value: 0.6194 vs 0.6113, respectively, p = 0.003).
As seen in Figure 1, following the elevation of standard for the number of minimal eLNs increasing from 2 to 13, the Harrell's C index for LNR category was always higher than that for pN category. The difference between the predictive capacities of the LNR categories and pN categories was significant when the number of minimal eLNs is from 2 to 6 (P<0.05) and the difference lost statistical significance when the number of minimal eLNs was from 7 to 13 (P>0.05). However, as the minimal eLNs increased from 14 to 21, the Harrell's C index for pN category was slightly higher than that for LNR category, but the difference between them was not significant(p>0.05).
The p value reflects the significance of comparison between pN categories and LNR categories using different standards for the minimal number of lymph nodes examined.
Using log-rank test, comparison of the survival rates among different LNR categories in different pN categories revealed that there were significant prognostic differences among patients in different pN categories for any LNR category (P<0.001; Fig. 2A, 2B, 2C, 2D). Furthermore, as seen in Figure 2E which reflects the prognostic hazard ratio based on a Cox proportional hazards model with LNR and LNs+ as covariates, at the same LNR level, following the elevation of LNs+, the prognostic hazard ratio increased. That also means there was significant survival heterogeneity among different pN categories at the same LNR category.
(a) Survival curves for patients of LNR1; (b) Survival curves for patients of LNR2; (c) Survival curves for patients of LNR3; (d) Survival curves for patients of LNR4; (e) Mesh plots reflect the predictive hazard ratio based on a Cox proportional hazards model with LNR and LNs+ as covariates. (f) Mesh plots with the green, yellow, and red planes which subgroup the hazard ratios into four risk levels (hazard ratio: <1.21, 1.21–1.62, 1.62–1.72 and >1.72).
A Cox proportional hazards regression with both LNR and LNs+ as covariates was run to calculate the prognostic hazard ratio (HR). After determining the parameters , the formula was:
. Then, we grouped the patients into four risk levels according to HR and formed the nLN category: nLN1 = an HR<1.21; nLN2 = an HR between 1.21 and 1.62; nLN3 = an HR between 1.62 and 2.72; and nLN4 = an HR>2.72 (Fig. 2F). Patients with different nLN categories were divided into: nLN1, 34.3% (7747/34476); nLN2, 36.0% (12395/34476); nLN3, 26.6% (9157/34476); and nLN4, 15.0% (5177/34476). Survival differences among the groups were statistically significant (P<0.001; Table 1). In the multivariate analysis, the nLN category was significantly correlated with prognosis.
Figures 3A, 3B and 3C display the survival curves based on three different category approaches: pN categories, LNR categories and our nLN categories. We compared the Nagelkerke R2 and Harrell's C among the three categories. As a result, the nLN category had the highest Nagelkerke R2 (pN categories: 0.063; LNR categories: 0.065; nLN categories: 0.072; Table 3). In addition, comparison of Harrell's C statistics and BIC also revealed that our nLN categories had a better predictive capacity than both pN categories and LNR categories (p<0.05; Table 3). Moreover, the results of comparison of cumulative AUC demonstrated that the nLN categories had a higher accuracy in survival prediction than both pN categories and LNR categories at all post-operation time points (Fig. 4).
(a) Survival curves for patients classified by AJCC pN categories; (b) Survival curves for patients classified by LNR categories; (c) Survival curves for patients classified by the novel categories (nLN).
Comparison of the survival rate between patients with ≥12 eLNs and <12 eLNs stratified by all three categories revealed that the heterogeneities of prognosis between patients with <12 eLNs and ≥12 eLNs at nLN categories was the lowest among three categories. In four pN categories there were significant survival differences between patients with <12 eLNs and ≥12 eLNs (5-year accumulative survival rates: 69.0% vs. 76.4% at N1a, p<0.001; 59.9% vs. 67.9% at N1b, p<0.001; 44.3% vs. 57.4% at N2a, p<0.001; 29.2% vs. 36.8% in N2b, p<0.001; Fig. 5A). Although the heterogeneities of prognosis between patients with <12 eLNs and ≥12 eLNs at LNR categories were lower than that of pN categories, there were significant survival heterogeneities in four LNR categories (5-year accumulative survival rates: 72.5% vs. 74.7% at LNR1, p = 0.017; 69.0% vs. 64.0% at LNR2, p<0.001; 59.1% vs. 52.3% at LNR3, p<0.001; 41.8% vs. 28.7% at LNR4, p<0.001; Fig. 5B). Conversely, there were no significant survival heterogeneities between patients with <12 eLNs and ≥12 eLNs at nLN2 and nLN3 (5-year accumulative survival rates: 67.4% vs. 66.6% at nLN2, p = 0.422; 54.3% vs. 52.7% at nLN3, p = 0.268). There were survival differences in nLN1 and nLN4 (5-year accumulative survival rates: 72.9% vs. 76.3% at nLN1, p = 0.001; 39.2% vs. 29.3% at nLN4, p<0.001; Fig. 5C).
(a) Survival curves stratified by AJCC pN categories; (b) Survival curves stratified by LNR categories; (c) Survival curves stratified by the novel categories (nLN). The blue lines represent the survival curves of patients with <12 eLNs and the yellow lines represent the survival curves of patients with ≥12 eLNs. The 5-year accumulative survival rates (5YSR) for patients were also presented.
Discussion
Over the past few years, LNR has been studied widely in the prognostic analysis of colon cancer. Nearly all researchers demonstrated that the LNR is an independent prognostic factor. However, it is still unclear whether the LNR category has more prognostic validity than the AJCC pN category [18], [19]. In our study, we compared the predictive capacity of the LNR category with that of pN based on the SEER dataset. We found that the LNR category was significantly better than the pN category in predictive value in whole groups of patients (Harrell's C index: 0.6194 vs 0.6113, respectively, p = 0.003). This result was similar to previous studies [14], [15], [17], [26], [27].
However, there is still debate on whether the LNR has more prognostic validity than the AJCC pN category if the eLNs is enough. Priolli et al. analyzed the prognostic value of the LNR in patients with no less than 12 eLNs and multivariate analysis showed that both the LNR and lymph node involvement were independent prognostic factors. Furthermore, lymph node involvement obtained a higher ‘score’ than the LNR [16]. Recently, based on the SEER dataset, Chen et al. compared the prognostic values of the LNR categories with that of the pN categories in patients with no less than 12 eLNs. Multivariate analysis showed that both LNR and lymph node involvement were independent prognostic factors. They proposed that the LNR categories had better prognostic value than the pN categories for the reason that the LNR categories had a higher hazard ratio than the pN categories [28]. In this study, we found that when the minimal eLNs was no less than 14, the predictive capacity of the pN categories was even higher than the LNR categories, although the difference is not significant statistically. That meant that LNR category was not superior to pN category in the predictive captivity all the time. Maybe this result could be affected by the cut-off value of the LNR categories, while the optimal cut-off value for LNRs has not received consensus [29] and the cut-off values used in this study were searched by statistic method to ensure the efficiency.
Moreover, it was verified that there was significant survival heterogeneity among different pN categories at the same LNR category. Using log-rank test, comparison of the survival rates among different LNR categories in different pN categories revealed that there were significant prognostic differences among patients in different pN categories for any LNR category (P<0.001; Fig. 2A, 2B, 2C, 2D). Therefore, it is not scientific enough if the pN category is simply replaced by LNR category. The result of Cox proportional hazards model with LNR and LNs+ as covariates also supported this opinion (Fig. 2E). Nevertheless, the prognostic value of LNR could not be ignored. Maybe a category which integrated the LNR with LNs+ is considerable.
In light of these considerations, a Cox proportional hazards regression with both LNR and LNs+ as covariates was run to calculate the prognostic hazard ratio (HR). After calculation of the parameters, the formula: was obtained. Both the LNR and LNs+ were referred by this formula and the large dataset used in this study make sure that the parameters 1.1875 and 0.0484 were accurate. And then, we divided calculated HR into four risk levels and formed our new category (nLN): nLN1 = an HR<1.21; nLN2 = an HR between 1.21 and 1.62; nLN3 = an HR between 1.62 and 2.72; and nLN4 = an HR>2.72.
Survival differences among the groups were statistically significant (P<0.001). Furthermore, using three statistical methods; i.e., Nagelkerke R2, Harrell's C and BIC, we verified the effectiveness of the nLN category and compared it with the LNR categories and pN categories, respectively. We found that the nLN category had higher predictive capacity than the other two categories (Table 3). Moreover, based on comparison of cumulative AUC, we found that the nLN categories had a higher accuracy in survival prediction than both pN categories and LNR categories at all post-operation time points (Fig. 4). Furthermore, compared with pN and LNR categories, the nLN category had more value in reduction of the heterogeneity of prognosis caused by insufficient eLNs. In the present study, we found significant heterogeneities of prognosis between patients with <12 eLNs and ≥12 eLNs at all four pN and LNR categories (Fig. 5A, 5B). Conversely, there was no heterogeneities of prognosis between patients with <12 eLNs and ≥12 eLNs at nLN2 and nLN3 of the nLN category (Fig. 5C). To some degree, the nLN category can be used to counterbalance incomplete nodal assessment for pathological evaluation and increase the accuracy of prognostic predication irrespective of the eLNs. These findings indicated that nLN categories were suitable for predicting the prognosis of patients with colon cancer. And then, based on the nLN category, the patients may obtain some clinical benefit from accurate prediction of long-term prognosis and appropriate adjuvant treatment planning.
Our study has some limitations. It is a retrospective exploratory study based on SEER data. Clinical and pathologic patient information can be heterogeneous since SEER collects information from 12 population-based cancer registries. On the other hand, data on adjuvant therapy is limited to information on radiation therapy only and it was reported that the total number of retrieved lymph nodes may decrease after preoperative chemotherapy [20]. Also, there is a lack of information on other factors that are related to the total number of retrieved lymph nodes, such as BMI index [28]. This makes some subgroup analysis impossible. Further, external validation by using other sources of data with sufficient pathological information is needed.
We conclude that, to evaluate the prognosis of colon cancer, our nLN category which intergrades LNR with LNs+ is more accurate than the pN category or LNR category, respectively.
Author Contributions
Conceived and designed the experiments: PG YXS ZNW. Performed the experiments: PG LLT. Analyzed the data: PG LLT. Contributed reagents/materials/analysis tools: YXS YYX JLZ QCT HMX. Wrote the paper: ZNW YYX JLZ QCT HMX.
References
- 1. Siegel R, Ward E, Brawley O, Jemal A (2011) Cancer statistics, 2011: the impact of eliminating socioeconomic and racial disparities on premature cancer deaths. CA Cancer J Clin 61: 212–236.
- 2. Greene FL (2007) Current TNM staging of colorectal cancer. Lancet Oncol 8: 572–573.
- 3.
Benson AB 3rd, Arnoletti JP, Bekaii-Saab T, Chan E, Chen YJ, et al. (2012) NCCN Clinical Practice Guidelines in Oncology: Colon Cancer. V.2.2012. Available: www.nccn.org. Accessed 2012 Jan 3.
- 4. Le Voyer TE, Sigurdson ER, Hanlon AL, Mayer RJ, Macdonald JS, et al. (2003) Colon cancer survival is associated with increasing number of lymph nodes analyzed: a secondary survey of intergroup trial INT-0089. J Clin Oncol 21: 2912–2919.
- 5. Hashiguchi Y, Hase K, Ueno H, Mochizuki H, Kajiwara Y, et al. (2010) Prognostic significance of the number of lymph nodes examined in colon cancer surgery: clinical application beyond simple measurement. Ann Surg 251: 872–881.
- 6.
Edge SB, Byrd DR, Compton CC, Fritz AG, Greene FL, et al. (2010) AJCC cancer staging manual, 7th ed. New York: Springer.
- 7. Compton CC, Fielding LP, Burgart LJ, Conley B, Cooper HS, et al. (2000) Prognostic factors in colorectal cancer. College of American Pathologists Consensus Statement 1999. Arch Pathol Lab Med 124: 979–994.
- 8. Downing SR, Cadogan KA, Ortega G, Jaji Z, Bolorunduro OB, et al. (2010) The number of lymph nodes examined debate in colon cancer: how much is enough? J Surg Res 163: 264–269.
- 9. Chen SL, Bilchik AJ (2006) More extensive nodal dissection improves survival for stages I to III of colon cancer: a population-based study. Ann Surg 244: 602–610.
- 10. Swanson RS, Compton CC, Stewart AK, Bland KI (2003) The prognosis of T3N0 colon cancer is dependent on the number of lymph nodes examined. Ann Surg Oncol 10: 65–71.
- 11. Berger AC, Sigurdson ER, LeVoyer T, Hanlon A, Mayer RJ, et al. (2005) Colon cancer survival is associated with decreasing ratio of metastatic to examined lymph nodes. J Clin Oncol 23: 8706–8712.
- 12. Rosenberg R, Engel J, Bruns C, Heitland W, Hermes N, et al. (2010) The prognostic value of lymph node ratio in a population-based collective of colorectal cancer patients. Ann Surg 251: 1070–1078.
- 13. Tong LL, Gao P, Wang ZN, Song YX, Xu YY, et al. (2011) Can lymph node ratio take the place of pN categories in the UICC/AJCC TNM classification system for colorectal cancer? Ann Surg Oncol 18: 2453–2460.
- 14. Derwinger K, Carlsson G, Gustavsson B (2008) A study of lymph node ratio as a prognostic marker in colon cancer. Eur J Surg Oncol 34: 771–775.
- 15. Rosenberg R, Friederichs J, Schuster T, Gertler R, Maak M, et al. (2008) Prognosis of patients with colorectal cancer is associated with lymph node ratio: a single-center analysis of 3,026 patients over a 25-year time period. Ann Surg 248: 968–978.
- 16. Priolli DG, Cardinalli IA, Pereira JA, Alfredo CH, Margarido NF, et al. (2009) Metastatic lymph node ratio as an independent prognostic variable in colorectal cancer: study of 113 patients. Tech Coloproctol 13: 113–121.
- 17. Vaccaro CA, Im V, Rossi GL, Quintana GO, Benati ML, et al. (2009) Lymph node ratio as prognosis factor for colon cancer treated by colorectal surgeons. Dis Colon Rectum 52: 1244–1250.
- 18. Ceelen W, Van Nieuwenhove Y, Pattyn P (2010) Prognostic value of the lymph node ratio in stage III colorectal cancer: a systematic review. Ann Surg Oncol 17: 2847–2855.
- 19. Noura S, Ohue M, Kano S, Shingai T, Yamada T, et al. (2010) Impact of metastatic lymph node ratio in node-positive colorectal cancer. World J Gastrointest Surg 2: 70–77.
- 20. Wong KP, Poon JT, Fan JK, Law WL (2011) Prognostic value of lymph node ratio in stage III colorectal cancer. Colorectal Dis 13: 1116–1122.
- 21. Newson RB (2010) Comparing the predictive powers of survival models using Harrell's C or Somers' D. Stata J 10: 339–358.
- 22. Harrell FE Jr, Lee KL, Mark DB (1996) Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 15: 361–387.
- 23. Nagelkerke N (1991) A note on a general definition of the coefficient of determination. Biometrika 78: 691–692.
- 24. Volinsky CT, Raftery AE (2000) Bayesian information criterion for censored survival models. Biometrics 56: 256–62.
- 25. Song X, Zhou XH (2008) A semiparametric approach for the covariate specific ROC curve with survival outcome. Statistica Sinica 18: 947–965.
- 26. Hong KD, Lee SI, Moon HY (2011) Lymph node ratio as determined by the 7th edition of the American Joint Committee on Cancer staging system predicts survival in stage III colon cancer. J Surg Oncol 103: 406–410.
- 27. Moug SJ, Saldanha JD, McGregor JR, Balsitis M, Diament RH (2009) Positive lymph node retrieval ratio optimises patient staging in colorectal cancer. British Journal of Cancer 100: 1530–1533.
- 28. Chen SL, Steele SR, Eberhardt J, Zhu K, Bilchik A, et al. (2011) Lymph Node Ratio as a Quality and Prognostic Indicator in Stage III Colon Cancer. Annals of Surgery 253: 82–87.
- 29. Huh JW, Kim YJ, Kim HR (2010) Ratio of metastatic to resected lymph nodes as a prognostic factor in node-positive colorectal cancer. Ann Surg Oncol 17: 2640–2646.