Figures
Abstract
Background
Numerous prognostic gene expression signatures have been recently described. Among the signatures there is variation in the constituent genes that are utilized. We aim to evaluate prognostic concordance among eight gene expression signatures, on a large dataset of ER positive HER2 negative breast cancers.
Methods
We analysed the performance of eight gene expression signatures on six different datasets of ER+ HER2- breast cancers. Survival analyses were performed using the Kaplan–Meier estimate of survival function. We assessed discrimination and concordance between the 8 signatures on survival and recurrence rates The Nottingham Prognostic Index (NPI) was used to to stratify the risk of recurrence/death.
Results
The discrimination ability of the whole signatures, showed fair discrimination performances, with AUC ranging from 0.64 (95%CI 0.55–0.73 for the 76-genes signatures, to 0.72 (95%CI 0.64–0.8) for the Molecular Prognosis Index T17. Low concordance was found in predicting events in the intermediate and high-risk group, as defined by the NPI. Low risk group was the only subgroup with a good signatures concordance.
Conclusion
Genomic signatures may be a good option to predict prognosis as most of them perform well at the population level. They exhibit, however, a high degree of discordance in the intermediate and high-risk groups. The major benefit that we could expect from gene expression signatures is the standardization of proliferation assessment.
Citation: Laas E, Mallon P, Duhoux FP, Hamidouche A, Rouzier R, Reyal F (2016) Low Concordance between Gene Expression Signatures in ER Positive HER2 Negative Breast Carcinoma Could Impair Their Clinical Application. PLoS ONE 11(2): e0148957. https://doi.org/10.1371/journal.pone.0148957
Editor: William B. Coleman, University of North Carolina School of Medicine, UNITED STATES
Received: October 25, 2015; Accepted: January 25, 2016; Published: February 19, 2016
Copyright: © 2016 Laas et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All.CEL files are available from the Gene Expression Omnibus and ArrayExpress repository websites (URLs: http://www.ebi.ac.uk/microarray-as/ae/ and http://www.ncbi.nlm.nih.gov/geo/). This dataset was previously used (doi:10.1186/bcr2192).
Funding: The authors have no support or funding to report.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Multi-gene expression assays as tools for clinical decision-making are increasingly being used in clinical practice. The 2011 St. Gallen International Expert Consensus [1] recognized the 21-gene signature (Oncotype DX, GenomicHealth) as a tool that "may be used" to predict chemotherapy responsiveness in patients with ER positive node negative breast tumors. The latest NCCN guidelines [2] include "multi-gene testing" in the list of factors used to select local and systemic therapies. They stated that the 21-gene signature may assist in evaluating patients for chemotherapy whose primary invasive breast tumors are 0.6–1.0 cm with unfavourable features or >1 cm and node-negative, ER/PR positive and HER2-negative.
The 21-gene assay (Oncotype DX), uses reverse transcription polymerase chain reaction (RT-PCR) on RNA isolated from paraffin-embedded breast cancer tissue. Two retrospective trials on Oncotype DX have quantified the risk of recurrence as a continuous variable and can predict response to tamoxifen [3] and chemotherapy [4] in patients with ER positive node negative disease. The Oncotype DX recurrence score assay also provides predictive information for chemotherapy benefit in patients with axillary lymph node-positive ER-positive breast cancer. The data in node-positive patients, however, was less robust as it relied on retrospective subset analysis of a single randomized trial [5].
The MammaPrint assay (70-gene expression profile) is approved by the FDA to assist in categorizing women younger than 61-years into a high vs. low risk of recurrence [6,7]. The Mindact trial8 will help conclude if Mammaprint can assist in selection of adjuvant chemotherapy for breast cancer patients with 0–3 nodes
Other multi-gene expression assay systems are currently not recommended for use in clinical practice but there is an increasing amount of literature reporting on their clinical relevance [8–14].
Previous studies have however raised concerns about concordance and stability of gene signatures for ER/PR/HER2 negative[15] and ER positive/HER2 negative cancers[16,17].Another concern is that expression levels of single and multi-gene signatures are unstable[18–20] In a previous study, we found that the concordance of outcome assignment between eight gene expression signatures was low [20]. Furthermore, a recent review by Engelhardt et al showed that the accuracy of genes signature prognostic estimates was suboptimal in some patient subgroups [21]. These signatures were designed to help oncologists make clinical decisions on the necessity of adjuvant therapy. It is important, therefore, to determine if current signatures are standardized and reproducible in different patient sub-groups.
In this study, we aim to evaluate the concordance in survival prediction of 8 gene-expression signatures on a large dataset of ER positive HER2 negative breast cancers.
Materials and Methods
Data pre-processing
As previously described, six breast cancer datasets [9,22–26] (all arrayed on the same platform [HGU-133A Affymetrix Santa Clara, California, USA] to avoid cross-platform discrepancies) for which the raw data (.CEL files) were publicly available were downloaded from the Gene Expression Omnibus and ArrayExpress repository websites [27,28]. This resulted in a total of 1,143 microarrays. Of these, 1,127 were deemed to be of sufficient quality and were kept in the current study. Sixteen micro-arrays were rejected due to major artifacts in the original.CEL files. To ensure comparability between the different datasets, they were all subjected to the same pre-processing procedure. Microarray quality-control assessment was carried out using the AffyPLM R-package [29]. Selected arrays were normalized using the RMA expression measure algorithm [30].
Determination of ER/Her2 status
The oestrogen receptor (ER) gene expression status was determined using the 205225_at probe set [31]. A gene-expression cut-off of 1.834 resulted in a 90% sensitivity in determining the ER status (IHC). All samples with a gene expression value higher than 1.834 were classified as ER positive. Similarly, the Her2 gene expression status was determined using the 216836_s_at probe set. A density plot of the 1,127 gene expression values showed a bimodal distribution. The lowest value of the density plot between the two modes determined the cut-off between the Her2 positive and Her2 negative status. All samples with a gene expression value higher than 1.62 were classified as Her2 positive.
Signature validation
On the complete dataset, we applied the 76-gene signature [8]; Chromosomal Instability Signatures [CIN70, CIN25] [11]; Core Serum Response signature [CSR] [12]; Invasiveness Gene Signature [IGS][13]; Molecular Prognosis Index signature [T52, T17] [10] and Gene expression Grade Index [GGI] [14]. These classifiers were defined as previously described[20].
Statistical analysis of the concordance of the 8 gene signatures
To identify robust subgroups of patients with different outcome we used the Nottingham Prognostic Index (NPI) score. It is based on a combination of three pathological criteria (tumour size, lymph-node involvement stage and tumour grading) assembled in a prognostic index formula [32]. It can be used as a continuous variable for risk stratification in unselected cohorts of operable early-stage primary breast cancer patients. Prognosis worsens as the NPI numerical value increases. It can also stratify patients into good, moderate and poor prognostic groups [33]. The NPI has been validated independently in large multicentre studies [34,35]. Because the precise number of positive axillary nodes was not known, we used a slightly different definition for the NPI, as described by Teschendorff et al [36]: NPI = 0.2 × (Tumor_Size [cm]) + Grade + 1.5 × (Node_Status) + 1 where Node Status can be positive (1) or negative (0) and Grade can be 1, 2 or 3. Scores lower than 3.4 defined the low risk group. A score between 3.4 and 5.4 defined intermediate risk group, whereas the high-risk group had a score higher than 5.4.To evaluate signatures performance, we performed survival analyses using the Kaplan–Meier estimate of the survival function. Comparison between survival curves was performed using the log-rank test. P values were considered significant when <0.05. Discrimination (i.e., whether the relative ranking of individual predictions was in the correct order) was quantified in both populations with the measure of the area under curve (AUC) ROC. For this analyze, the outcome was dichotomized into a poor outcome group (samples with an event within 5 years of follow-up) and a good outcome group (samples with no event and a follow-up of at least 5 years. Events were defined as distant metastases occurrence or breast cancer specific death. Concordance was first strictly defined as 0 or 8 poor prognosis signatures. Then, we observed concordance with a broader definition: 0 or 1 and 7 or 8 poor signatures.
We observed the distribution of discordant signatures predictions, according to the NPI stratification (low risk, intermediate risk, high risk).
The analysis was performed using R software using the survival, rms and pROC package (http://cran.r-project.org/).
Results
Population
From our previous study, we identified 769 patients out of the 1,127 presenting an ER positive HER2 negative breast carcinoma. For 454 samples, distant metastasis-free survival (DMFS) data were available; while for 349 samples, breast cancer-specific survival (BCSS) data were available. The clinical and pathological features are described in Table 1.
NPI Prognostic performance
The NPI score was able to identify in our population three patients subgroups with significantly different outcome. 22.5% were classified as low risk (13.2% of events), 57.3% were classified as intermediate risk (21.7% of events) and 20.2% were classified as high risk (36.7% of events). The 5 years survival rate (BCSS and DMFS) was significantly different between groups (Log rank pvalue <0.001): 98% in the low-risk group, 92% in the intermediate-risk group (HR 4.67) and 66.7% in the high-risk group (HR 20.03) (Fig 1).
Gene expression Signatures performance
The percentage of patients with a “poor prognosis” call for each signature varies between 30 and 68%.
For the BCSS, the performance in terms of log-rank test p-value was similar between the different molecular signatures (Fig 2A), with hazard ratios ranging from 2.4 for the CIN25, CIN70 and GGI signatures to 4.5 for the CSR signature (Table 2).
(A)Breast Cancer Specific Survival (Kaplan Meier curves) (B) Discrimination performances (ROC curves)
For the DMFS, the performance was also similar between the different molecular signatures with hazard ratios ranging from 1.4 for the Wang signature to 2.2 for the IGS signature (Table 2).
Similarly, the discrimination ability of the whole signatures for both DMFS and BCSS, measured using the AUC, showed fair discrimination performances. The AUC ranged from 0.64 for the Wang signature to 0.72 for the TESC17 signature, with overlapped confidence interval (Fig 2B).
However, from an individual viewpoint, 39.9% of the events failed to be predicted by none or less than 5 signatures (Table 3).
Signature concordance and accuracy
When concordance was defined as none (0) or 8 signatures with similar outcome assignments, more than 50% of the population (51.5%) had a discordant prediction. 53% of all events were identified in this subgroup (Table 3). The discordance rates were similar between the three risk subgroups as defined by the NPI score: low-risk group 52.1%, intermediate-risk group 53.1% and high-risk group 51.4% (p = 0.9).
With a broader concordance definition (0–1, 7–8 poor outcome signatures), 25.8% of the population had a discordant prediction, with only 2 to 6 signatures with similar outcome assignments. 27.4% of all events were identified in this subgroup (Table 3). The low-risk group, as defined by a NPI score below 3.4, had the highest concordance between signatures, with 66.1% of concordant good prognosis prediction (0 or 1 poor signature), and 23.1% of discordant signatures (2 to 6). The discrepancy rate was higher in the intermediate and high-risk subgroup, with 26.2% and 31.2% of signatures discordances respectively (p = 0.4) (Fig 3).
*We considered concordance for 0 or 1 signature and 7 or 8 signatures.
Impact of gene-signature number on concordance
We calculated the concordance for every combination of 2 (28 combinations), 3 (56 combinations), 4(70 combinations), 5 (56 combinations), 6 (28 combinations), and 7 (8 combinations) gene-expression signatures out of the 8 available in this study. The median concordance rate decreased significantly with the number of signature available: from 79.4% for 2 signatures to 48.5% if we used 8 signatures (Fig 4A). Regardless of the number of signatures available (2 to 8), the concordance was consistently better in the low-risk group than in the intermediate-risk group and high-risk group (p value = 1) (Fig 4B).
Combination of 2 to 7 molecular signatures. (A)Overall population (B) NPI risk groups.
Discussion
We observed that 8 gene signatures consistently identified a small subgroup of ER-positive HER2-negative breast tumors patients with an excellent prognosis. In this subgroup, the risk of distant events remains low (4–6%) and would eventually justify a systemic treatment with tamoxifen or aromatase inhibitors alone. In addition, the 8 gene signatures consistently identified a small subgroup of patients with the worst prognosis. This however did not translate to clinical outcome, as 65% of this patient group did not develop distant metastases, or disease specific death. Between these two poles of high concordance we identified a large “grey zone” were gene-expression signatures did not reach a consensus in the prediction of events. The discordance increased with the number of signature used. This heterogeneity may be attributed to the different methodologies that were followed to build the classifiers, the heterogeneity in the sample populations used to build the classifiers, and the variation in sample size s[37]. Many genes signatures exist and many more will be available over the years with little agreement in the constituent genes. Wirapati et al showed that the prognostic abilities of nine gene expression signatures would be due mostly to the detection of proliferation activity [15]. However, few studies have investigated the concordance between existent predictors. Fan et al have shown significant agreement (81%) in the outcome predictions of five genomic signatures (including Mammaprint and OncotypeDX), pairwise compared [38]. In another study from Haibe-Kains et al, similar prognostic performances were found between the 70-gene, the 76-gene and the Gene expression Grade Index signatures (68% when considering the three signatures, 71% for the 70- and 76-gene signatures, 76% for the 76-gene signature and the GGI, and 88% for the 70-gene signature and the GGI). [39].
Several studies have described methods to improve the stability of gene signatures. One way is to increase the sample size. It generates a gene signature with better concordance in outcome prediction and better prediction accuracy [40]. Other statistical methods (resampling, bootstrap) can improve stability and performance in some cases. Another way of improving the stability of a gene signature suggested by Zhao et al was to combine the information obtained from published gene-sets prognosis signatures to build a higher prognostic performance signature using an independent gene expression dataset [41]. In a previous study, Reyal et al showed that the combination of the signatures could define an efficient classifier for breast cancer, which will most probably be more stable than the classifiers from which it originates [20].
It is therefore not surprising that the risks predicted by these models are in the end little used in clinical setting for adjuvant therapies prescription. In the review from Hornberger et al, use of risk prediction model (comprising Oncotype DX, Mammaprint or Adjuvant Online) led to a change in treatment recommendation ranging from 1% to 74% [42], but with considerable heterogeneity between studies.
It is thus debatable whether these expensive tools add significant information to the clinical and biological data already available,. The use of Ki67 may increase in popularity as one of its major drawbacks, the lack of standardization, has been addressed by recent international recommendations [43]. Refining outcome prediction by adding other immunohistochemical data like TP53 expression is another route that should be explored and that might prove more cost-effective than the use of gene expression signatures [44].
New models based on genetic profiles combined with clinical factors, may further improve prediction. Tang et al have associated the OncotypeDx recurrence score to clinicopathologic factors in the Recurrence Score Pathology-Clinical (RSPC) score. This model substantially reduces the number of patients classified in the intermediate risk category in the assessment of distant recurrence risk [45]. Similarly, the immunohistochemical (IHC) 4+C score is a cost-effective prognostic tool that uses clinicopathologic factors and four standard IHC assays: oestrogen receptor, progesterone receptor, HER2 and Ki67 [46]. It has been shown to substantially improve decision-making on adjuvant chemotherapy [47]. Obviously, concordance of these predictions should be evaluated too, since improvement of prediction does not mean improvement in concordance of prediction.
One of the major limitations of our study is to not integrate the two well-known signatures that are currently the most widely used and validated: the 21-gene Genomic Health signature OncotypeDX [3] and the 70-gene signature from the Netherlands Cancer Institute (Mammaprint) [7]. The Genomic Health signature was not included since it is not a micro-array-based gene expression signature but is designed for RT-PCR assays. The 70-gene signature was not included since the 295 samples dataset on which this signature was developed was employed as a completely independent validation set.
In conclusion, regarding the adverse events and costs of adjuvant therapies, reliable decision assisting tools must be developed. Genomic signatures could be a good option, but this study revealed that if most genomic signatures perform well at the population level, they exhibit a high degree of discordance in the intermediate and high-risk groups. Moreover, in the high-risk group, there were only 38.5% of events when the 8 signatures were concordant for a poor prognosis, adding to the measurement error. Clinical as well as genomic prediction models in ER positive HER2 negative breast cancer subgroups are only able to reliably identify good prognosis patients. The main argument to support the use of genomic predictors would be a single standardized gene signature tool. Unfortunately, the current high level of discordance between gene signatures complicates their use for routine clinical practice. We have demonstrated in this study that the use of multiple genomic signatures in ER-positive HER2-negative breast tumours affects the accuracy of prognosis prediction.
Author Contributions
Conceived and designed the experiments: EL RR FR. Performed the experiments: EL AH. Analyzed the data: EL FR. Contributed reagents/materials/analysis tools: EL PM FPD. Wrote the paper: EL PM AH.
References
- 1. Goldhirsch A, Wood WC, Coates AS, Gelber RD, Thürlimann B, Senn H- J, et al. Strategies for subtypes—dealing with the diversity of breast cancer: highlights of the St. Gallen International Expert Consensus on the Primary Therapy of Early Breast Cancer 2011. Ann Oncol Off J Eur Soc Med Oncol ESMO. 2011;22: 1736–1747.
- 2.
National Comprehensive Cancer Network [Internet]. Available: https://www.nccn.org/store/login/login.aspx?ReturnURL=http://www.nccn.org/professionals/physician_gls/pdf/breast.pdf.
- 3. Paik S, Shak S, Tang G, Kim C, Baker J, Cronin M, et al. A multigene assay to predict recurrence of tamoxifen-treated, node-negative breast cancer. N Engl J Med. 2004;351: 2817–2826. pmid:15591335
- 4. Paik S, Tang G, Shak S, Kim C, Baker J, Kim W, et al. Gene expression and benefit of chemotherapy in women with node-negative, estrogen receptor-positive breast cancer. J Clin Oncol Off J Am Soc Clin Oncol. 2006;24: 3726–3734.
- 5. Albain KS, Barlow WE, Shak S, Hortobagyi GN, Livingston RB, Yeh I-T, et al. Prognostic and predictive value of the 21-gene recurrence score assay in postmenopausal women with node-positive, oestrogen-receptor-positive breast cancer on chemotherapy: a retrospective analysis of a randomised trial. Lancet Oncol. 2010;11: 55–65. pmid:20005174
- 6. van de Vijver MJ, He YD, van’t Veer LJ, Dai H, Hart AAM, Voskuil DW, et al. A gene-expression signature as a predictor of survival in breast cancer. N Engl J Med. 2002;347: 1999–2009. pmid:12490681
- 7. van ‘t Veer LJ, Dai H, van de Vijver MJ, He YD, Hart AAM, Mao M, et al. Gene expression profiling predicts clinical outcome of breast cancer. Nature. 2002;415: 530–536. pmid:11823860
- 8. Wang Y, Klijn JGM, Zhang Y, Sieuwerts AM, Look MP, Yang F, et al. Gene-expression profiles to predict distant metastasis of lymph-node-negative primary breast cancer. Lancet. 2005;365: 671–679. pmid:15721472
- 9. Desmedt C, Piette F, Loi S, Wang Y, Lallemand F, Haibe-Kains B, et al. Strong time dependence of the 76-gene prognostic signature for node-negative breast cancer patients in the TRANSBIG multicenter independent validation series. Clin Cancer Res Off J Am Assoc Cancer Res. 2007;13: 3207–3214.
- 10. Teschendorff AE, Miremadi A, Pinder SE, Ellis IO, Caldas C. An immune response gene expression module identifies a good prognosis subtype in estrogen receptor negative breast cancer. Genome Biol. 2007;8: R157. pmid:17683518
- 11. Carter SL, Eklund AC, Kohane IS, Harris LN, Szallasi Z. A signature of chromosomal instability inferred from gene expression profiles predicts clinical outcome in multiple human cancers. Nat Genet. 2006;38: 1043–1048. pmid:16921376
- 12. Chang HY, Nuyten DSA, Sneddon JB, Hastie T, Tibshirani R, Sørlie T, et al. Robustness, scalability, and integration of a wound-response gene expression signature in predicting breast cancer survival. Proc Natl Acad Sci U S A. 2005;102: 3738–3743. pmid:15701700
- 13. Liu R, Wang X, Chen GY, Dalerba P, Gurney A, Hoey T, et al. The prognostic role of a gene signature from tumorigenic breast-cancer cells. N Engl J Med. 2007;356: 217–226. pmid:17229949
- 14. Sotiriou C, Wirapati P, Loi S, Harris A, Fox S, Smeds J, et al. Gene expression profiling in breast cancer: understanding the molecular basis of histologic grade to improve prognosis. J Natl Cancer Inst. 2006;98: 262–272. pmid:16478745
- 15. Wirapati P, Sotiriou C, Kunkel S, Farmer P, Pradervand S, Haibe-Kains B, et al. Meta-analysis of gene expression profiles in breast cancer: toward a unified understanding of breast cancer subtyping and prognosis signatures. Breast Cancer Res BCR. 2008;10: R65. pmid:18662380
- 16. Barry WT, Kernagis DN, Dressman HK, Griffis RJ, Hunter JD, Olson JA, et al. Intratumor Heterogeneity and Precision of Microarray-Based Predictors of Breast Cancer Biology and Clinical Outcome. J Clin Oncol. 2010;28: 2198–2206. pmid:20368555
- 17.
Haury A-C, Jacob L, Vert J-P. Increasing stability and interpretability of gene expression signatures. ArXiv Prepr ArXiv10013109. 2010; Available: http://arxiv.org/abs/1001.3109.
- 18. Michiels S, Koscielny S, Hill C. Prediction of cancer outcome with microarrays: a multiple random validation strategy. Lancet. 2005;365: 488–492. pmid:15705458
- 19. Ein-Dor L, Kela I, Getz G, Givol D, Domany E. Outcome signature genes in breast cancer: is there a unique set? Bioinforma Oxf Engl. 2005;21: 171–178.
- 20. Reyal F, Van Vliet MH, Armstrong NJ, Horlings HM, de Visser KE, Kok M, et al. A comprehensive analysis of prognostic signatures reveals the high predictive capacity of the proliferation, immune response and RNA splicing modules in breast cancer. Breast Cancer Res. 2008;10: R93. pmid:19014521
- 21. Engelhardt EG, Garvelink MM, de Haes JHCJM, van der Hoeven JJM, Smets EMA, Pieterse AH, et al. Predicting and communicating the risk of recurrence and death in women with early-stage breast cancer: a systematic review of risk prediction models. J Clin Oncol Off J Am Soc Clin Oncol. 2014;32: 238–250.
- 22. Loi S, Haibe-Kains B, Desmedt C, Lallemand F, Tutt AM, Gillet C, et al. Definition of clinically distinct molecular subtypes in estrogen receptor-positive breast carcinomas through genomic grade. J Clin Oncol Off J Am Soc Clin Oncol. 2007;25: 1239–1246.
- 23. Miller LD, Smeds J, George J, Vega VB, Vergara L, Ploner A, et al. An expression signature for p53 status in human breast cancer predicts mutation status, transcriptional effects, and patient survival. Proc Natl Acad Sci U S A. 2005;102: 13550–13555. pmid:16141321
- 24. Minn AJ, Gupta GP, Siegel PM, Bos PD, Shu W, Giri DD, et al. Genes that mediate breast cancer metastasis to lung. Nature. 2005;436: 518–524. pmid:16049480
- 25. Pawitan Y, Bjöhle J, Amler L, Borg A- L, Egyhazi S, Hall P, et al. Gene expression profiling spares early breast cancer patients from adjuvant therapy: derived and validated in two population-based cohorts. Breast Cancer Res BCR. 2005;7: R953–964. pmid:16280042
- 26. Chin K, DeVries S, Fridlyand J, Spellman PT, Roydasgupta R, Kuo W-L, et al. Genomic and transcriptional aberrations linked to breast cancer pathophysiologies. Cancer Cell. 2006;10: 529–541. pmid:17157792
- 27.
Gene Expression Omnibus [Internet]. Available: http://www.ncbi.nlm.nih.gov/geo/.
- 28.
ArrayExpress repository [Internet]. Available: http://www.ebi.ac.uk/microarray-as/ae/.
- 29.
R [Internet]. Available: http://bioconductor.wustl.edu/bioc/html/affyPLM.html.
- 30.
Bioconductor [Internet]. Available: http://www.bioconductor.org.
- 31. Gong Y, Yan K, Lin F, Anderson K, Sotiriou C, Andre F, et al. Determination of oestrogen-receptor status and ERBB2 status of breast carcinoma: a gene-expression profiling study. Lancet Oncol. 2007;8: 203–211. pmid:17329190
- 32. Haybittle JL, Blamey RW, Elston CW, Johnson J, Doyle PJ, Campbell FC, et al. A prognostic index in primary breast cancer. Br J Cancer. 1982;45: 361–366. pmid:7073932
- 33. Blamey RW, Ellis IO, Pinder SE, Lee AHS, Macmillan RD, Morgan DAL, et al. Survival of invasive breast cancer according to the Nottingham Prognostic Index in cases diagnosed in 1990–1999. Eur J Cancer Oxf Engl 1990. 2007;43: 1548–1555.
- 34. Brown J, Jones M, Benson EA. Comment on the Nottingham Prognostic Index. Breast Cancer Res Treat. 1993;25: 283. pmid:8267734
- 35. Balslev I, Axelsson CK, Zedeler K, Rasmussen BB, Carstensen B, Mouridsen HT. The Nottingham Prognostic Index applied to 9,149 patients from the studies of the Danish Breast Cancer Cooperative Group (DBCG). Breast Cancer Res Treat. 1994;32: 281–290. pmid:7865856
- 36. Teschendorff AE, Naderi A, Barbosa-Morais NL, Pinder SE, Ellis IO, Aparicio S, et al. A consensus prognostic gene expression classifier for ER positive breast cancer. Genome Biol. 2006;7: R101. pmid:17076897
- 37. van Vliet MH, Reyal F, Horlings HM, van de Vijver MJ, Reinders MJT, Wessels LFA. Pooling breast cancer datasets has a synergetic effect on classification performance and improves signature stability. BMC Genomics. 2008;9: 375. pmid:18684329
- 38. Fan C, Oh DS, Wessels L, Weigelt B, Nuyten DSA, Nobel AB, et al. Concordance among gene-expression-based predictors for breast cancer. N Engl J Med. 2006;355: 560–569. pmid:16899776
- 39. Haibe-Kains B, Desmedt C, Piette F, Buyse M, Cardoso F, Van’t Veer L, et al. Comparison of prognostic gene expression signatures for breast cancer. BMC Genomics. 2008;9: 394. pmid:18717985
- 40. Kim S-Y. Effects of sample size on robustness and prediction accuracy of a prognostic gene signature. BMC Bioinformatics. 2009;10: 147. pmid:19445687
- 41. Zhao X, Rødland EA, Sørlie T, Naume B, Langerød A, Frigessi A, et al. Combining gene signatures improves prediction of breast cancer survival. PloS One. 2011;6: e17845. pmid:21423775
- 42. Hornberger J, Alvarado MD, Rebecca C, Gutierrez HR, Yu TM, Gradishar WJ. Clinical validity/utility, change in practice patterns, and economic implications of risk stratifiers to predict outcomes for early-stage breast cancer: a systematic review. J Natl Cancer Inst. 2012;104: 1068–1079. pmid:22767204
- 43. Dowsett M, Nielsen TO, A’Hern R, Bartlett J, Coombes RC, Cuzick J, et al. Assessment of Ki67 in breast cancer: recommendations from the International Ki67 in Breast Cancer working group. J Natl Cancer Inst. 2011;103: 1656–1664. pmid:21960707
- 44. Millar EKA, Graham PH, McNeil CM, Browne L, O’Toole SA, Boulghourjian A, et al. Prediction of outcome of early ER+ breast cancer is improved using a biomarker panel, which includes Ki-67 and p53. Br J Cancer. 2011;105: 272–280. pmid:21712826
- 45. Tang G, Cuzick J, Costantino JP, Dowsett M, Forbes JF, Crager M, et al. Risk of recurrence and chemotherapy benefit for patients with node-negative, estrogen receptor-positive breast cancer: recurrence score alone and integrated with pathologic and clinical factors. J Clin Oncol Off J Am Soc Clin Oncol. 2011;29: 4365–4372.
- 46. Cuzick J, Dowsett M, Pineda S, Wale C, Salter J, Quinn E, et al. Prognostic value of a combined estrogen receptor, progesterone receptor, Ki-67, and human epidermal growth factor receptor 2 immunohistochemical score and comparison with the Genomic Health recurrence score in early breast cancer. J Clin Oncol Off J Am Soc Clin Oncol. 2011;29: 4273–4278.
- 47. Barton S, Zabaglo L, A’Hern R, Turner N, Ferguson T, O’Neill S, et al. Assessment of the contribution of the IHC4+C score to decision making in clinical practice in early breast cancer. Br J Cancer. 2012;106: 1760–1765. pmid:22531639