Data integration

The METABRIC dataset (n = 1904 patients, m = 18543 transcripts) was used to develop the prototype network implementation. Transcriptome data (median Z-scores), overall survival (OS), disease specific survival (DSS) and associated clinical records were downloaded from cbioportal.org [31,32]. The dataset was integrated with locoregional and distant recurrence information retrieved from Rueda et al. [33] and Risk of Recurrence—Proliferation (ROR-P) scores reported by Xia et al [9]. Clinical records, OS, DSS and progression free interval (PFI) of the validation TCGA-BRCA cohort (n = 1101 patients, m = 20532 transcripts) were integrated from cbioportal.org [31,32] and Liu et al. 2018 [34], respectively. To merge the mRNA expression data of the two cohorts, normalized transcriptome datasets were downloaded using the R package MetaGxBreast [35]. The transcript amounts were rescaled as described by Gendoo et al. [35] so that the 2.5 percentile corresponds to -1 and the 97.5 percentile corresponds to +1. Subsequently, transcripts overlapping between the two cohorts and with quantitative information missing in not more than five patients were retained. The resulting “post-merging” MB + TCGA dataset contains the expression values of m = 14042 genes. The R script used to download and rescale the datasets is available in the Zenodo repository [36].

Inclusion criteria and category definition

Both cohorts were filtered to exclude normal-like subtype samples [9,37,38] and patients for which less than 10 years of follow-up time from diagnosis were available. Low and high risk categories were defined according to published clinical records [8,9] as follows:

• high risk patients:
  • MB cohort: disease specific death, locoregional or distant recurrence event recorded before 10 years from initial diagnosis;
  • TCGA cohort: disease specific death, progression, local recurrence or distant metastases before 10 years from initial diagnosis.
• low risk patients: none of the above-mentioned events recorded before 10 years from initial diagnosis.

A follow-up time of 10 years is sufficient to capture both early and late relapse events [39]. In addition, the chosen clinical endpoints allow a direct comparison of the classifiers with the ROR-P categories of the MB cohort published by Xia et al [9].

In total, 1248 patients of the MB cohort (n = 567 high risk, n = 681 low risk) and 165 patients of the TCGA cohort (n = 132 high risk, n = 33 low risk) satisfied the inclusion criteria. Four patients from each cohort were excluded after merging the two datasets due to insufficient expression data.

GAN architecture

The architecture was based on a Wasserstein [40] GAN [21] with gradient penalty [41] and an auxiliary classifier [42] as a variant of a conditional GAN implementation [43], yielding a AC-WGAN-GP architecture. The Wasserstein loss was implemented to reduce vanishing gradients and mode collapse [44] in the early phases of the training when the discriminator outperformed the generator. Stability was improved by exchanging the weights clipping approach described in Arjovsky et al. [40], with the gradient penalty described in Gulrajani et al. [41]. To create a conditional GAN, an auxiliary classifier network was implemented [42], resulting in a more stable training process and reduced mode collapse compared to the standard conditional GAN approach, supplying labels to both discriminator and generator [44]. A z-vector of size 250 was fed as input for the generator. Following good training practice [45], strided convolutions with step size 2, batch normalization and LeakyRELU as activation function were used. Since using batch normalization in the discriminator and/or the ADAM optimizer led to an unstable training process, batch normalization [46] was only used in the generator, and RMSprop was selected as the activation function. A shallow network consisting of two layers in both the discriminator and the generator led to the most stable training process, due to the smaller number of trainable parameters compared to deeper networks. Hyperparameters were tuned empirically, selecting 1000 epochs for the training process. Three “discriminator-only” training runs were performed before each full network training run, and the generated pictures were subsequently smoothed with a final convolution layer with one filter and stride size of 1. The GAN architecture generated expression profiles of size 144 ×144 when using the MB dataset before merging the cohorts (m = 18453 transcripts), starting from a square matrix of size 18 × 18. After merging the MB and TCGA cohorts (m = 14042 overlapping transcripts), a starting matrix of size 15 × 15 was used, generating expression profiles of size 120 ×120. In both implementations, the values missing to populate the final square matrix were filled with random values sampled from a -1 to 1 uniform distribution. The resulting trained GAN Discriminator (T-GAN-D) was then used as an independent classifier to discriminate low and high risk patients. The Python code and the input files used to generate the predictions are available in the Zenodo repository [36].

CNN architecture

As the performance of the CNN implemented as the GAN’s discriminator showed satisfactory performance, a similar architecture was used as a benchmark classifier. Batch normalization was employed to ensure shorter training periods and RELU was used as the activation function. A fixed training length of 1250 epochs was set due to the limited sample size and to generate comparable iterations.

Feature selection and application of classic machine learning (ML) algorithms

Random Forest (RF) and Support Vector Machines (SVM) classifiers were implemented in the WEKA Workbench (Version 3.8) [47] using the classifiers trees.RandomForest and functions.LibSVM with default settings. Both algorithms were trained on the merged MB + TCGA cohort to predict the risk class of MB patients. A 5-fold cross validation was applied using the same training and test set splits to ensure comparability between the different methods. At each iteration, the attribute selection SymmetricalUncertAttributeSetEval with FCBFSearch (Fast Correlation-Based Filter [48]) method was applied to the training set. This preprocessing step aimed at selecting an optimal subset of transcripts with the highest correlation with the class and the lowest redundancy with other relevant features.

Survival analysis and accuracy

Log-rank testing was used to compare predicted low vs high risk patients over a follow-up time of 10 years. Kaplan-Meier (KM) survival curves were computed using GraphPad Prism 8 (GraphPad Software, San Diego, CA). The area between the curves (ABC) displayed on the KM graphs for the pooled predictions was calculated as follows:

• Low risk AUC minus Predicted low risk AUC;
• Predicted low risk AUC minus Predicted high risk AUC;
• Predicted high risk AUC minus High risk AUC.

The ABCs values are shown on the graphs in the abovementioned order top to bottom. The AUC was computed using GraphPad Prism 8 (GraphPad Software, San Diego, CA). Univariate and multivariate hazard ratios were calculated using the function coxph from the R’s library survival (v. 3.4.0, https://www.r-project.org/).

The accuracy of all classifiers was calculated dividing the number of correct classifications by the total number of classifications performed.

Sensitivity and specificity are calculated as follows: True positives are high risk individuals correctly classified as high risk; false positives are low risk individuals incorrectly classified as high risk; true negatives are low risk patients correctly classified as low risk; false negatives are high risk individuals incorrectly classified as low risk.

The METABRIC and TCGA-BRCA cohorts lend themselves as use cases for the application of GAN-based prognostic classifiers

One of the major challenges of machine learning applied to -omics data and companion medical records is the imbalance between the high amount of variables compared to the limited number of patients available. Even in the case of breast cancer, one of the most frequent and widely studied malignant neoplasms, this limitation is apparent in the two major public transcriptome datasets, namely the MB cohort (n = 1904 patients, m = 18543 transcripts) and the TCGA cohort (n = 1101 patients, m = 20532 transcripts). This imbalance is exacerbated for prognostic analyses that require long-term (10 years) follow-up information and the application of further exclusion criteria (see methods), reducing cohort sizes to n = 1248 and n = 165, respectively (Fig 1A and 1B). Both cohorts behaved notably different, with patients in the MB cohort on average having an overall substantially better prognosis in overall survival and relapse-free, progression-free or disease specific survival (Fig 1C and 1D). This is likely attributable to the MB dataset largely consisting of stage I and stage II patients (89.5% of patients with reported disease stage at diagnosis), whereas stage III and IV patients are more prominent in the TCGA dataset (40.4% of individuals with available disease stage at diagnosis). Despite these differences, the high risk subgroups of both cohorts showed comparable median survival times (MB = 31.9 months [Fig 1E], TCGA = 26.3 months [Fig 1F]). Due to the limited sizes of these cohorts, they lend themselves as suitably challenging use-cases for applying data augmentation for improving prognostication. In particular, we set out to implement a classifier based on a data augmentation network for improved patient stratification in the MB cohort, to subsequently validate robustness and transferability by integrating the independent TCGA cohort.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. MB and TCGA patient demographics and survival.

(A) Patient demographics of the MB subcohort. (B) Patient demographics of the TCGA subcohort. (C) Overall and (D) relapse-free, progression-free or disease specific survival of the MB and TCGA cohorts. (E) Kaplan Meier curves comparing low vs high risk patients of the MB and (F) the TCGA cohorts.

https://doi.org/10.1371/journal.pcbi.1011035.g001

A trained GAN discriminator robustly identifies low and high risk breast cancer patients

To tackle the problem of data scarcity, we implemented a GAN architecture to augment transcriptomic data of the MB cohort and tested the performance of a trained discriminator in stratifying breast cancer patients. First, individual patient transcriptome profiles were rescaled and converted into arrays of pixels (Fig 2A i) in order to use these images as an input for the GAN. Independent of these true patient data, the generator created images representing the transcript profiles of synthetic hypothetical patients together with their category (low or high risk) (Fig 2A ii). After being exposed to a fraction of the real transcriptome images and associated categories, its adversary, the discriminator network then tried to distinguish fake from real transcriptome images and determine the risk category of the synthetic samples (Fig 2A iii). Reiterating this training process over 1000 epochs, the generator learned to create realistic synthetic transcriptome images for high and low risk categories, which then could be used to augment the original training data. Associated characteristics of this process (discriminator loss, discriminator class loss, generator loss) are shown in S1 Fig. Using this approach, the discriminator learned to identify features relevant for the risk category definition, aided by the synthetic profiles that implicitly augmented the real training data at each epoch. The trained GAN discriminator (T-GAN-D) network resulting from this process was frozen and subsequently used as a standalone classifier in a transfer learning fashion. The T-GAN-D categorizes images from the test fraction of the cohort into the high or low risk categories (Fig 2A iv), thus prognosticating patient outcome. Afterwards, depending on the predicted class, patients were split into high and low risk sub-cohorts which were compared by survival analysis. The generated Log-rank P values were used as the main indicator to assess the stratification performance of the classifier.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. The T-GAN-D robustly stratifies low and high risk breast cancer patients.

(A) Workflow of the data processing, including the schematics of the generator network and its adversary, the discriminator network. Together these result in an AC-WGAN-GP architecture. After the conversion of patient transcriptome profiles into images, 4/5 of the MB dataset was used to train the GAN’s discriminator. After 1000 epochs, the trained discriminator was used as a standalone classifier to separate the remaining 1/5 patients of the dataset into low and high risk categories. (B) Kaplan-Meier curves separating low vs. high risk patients as predicted with the T-GAN-D (iteration 1 of the 5-fold CV shown as representative). (C) Kaplan-Meier curves generated pooling the category predictions obtained for all patients of the MB dataset after five independent CV runs. (D) Separation of low vs. high risk patients predicted with a classical CNN on the same subset used in B and (E) comparison obtained pooling the predictions of five independent CV runs. The area between the curves (ABC) between Low risk (blue dashed line) and Predicted low risk (solid blue line), Predicted low risk and Predicted high risk (solid red line), Predicted high risk and High risk groups (dashed red line) are shown top to bottom in D and E.

https://doi.org/10.1371/journal.pcbi.1011035.g002

We first implemented and tested the T-GAN-D for its prognostic capability using follow-up and mRNA expression data of the prototyping MB cohort, consisting of n = 1248 individuals and m = 18543 genes. Within this cohort, we independently cross-validated (CV) five-fold with randomly composed training data. Kaplan-Meier curves and log rank testing for each run yielded significant class separations in 4 out of 5 iterations (Figs 2B and S2A). Pooling the results so that each patient of the MB dataset was present once in the survival analysis, the T-GAN-D separated high and low risk patients with high statistical significance (p-value = 2.71E-12) (Fig 2C). To obtain a reference performance baseline, a classical CNN was challenged with the same task, using the same training and test sets for each iteration. The CNN yielded class separations with a p<0.05 in only two out of five iterations (Figs 2D and S2B). In the pooled comparison, the CNN performed well yet failed to outperform the T-GAN-D in separating low vs. high risk patients (Fig 2E and S1 Table). Taken together, these results demonstrate that the reiterative learning process of a GAN to train its discriminator and its use as an independent classifier provides a more robust and slightly improved patient stratification than a classical DL approach.

Introducing an independent cohort improves MB patient classification

A common limitation of predictors and classifiers is their limited robustness and transferability to independent datasets. This might arise from overfitting or overtraining within the initial cohort but also from heterogeneity and batch effects between source datasets. For validating our approach further, we therefore merged the mRNA expression data of the MB and TCGA cohorts, which originally were quantified with bead-based microarray technology (Illumina Human V3) or RNA-Seq (Illumina HiSeq) platforms respectively [49], by rescaling the expression of transcripts overlapping between the two cohorts (m = 14042). Both the generator and discriminator networks then were retrained using the entire TCGA data plus a fraction of the MB data from the merged dataset and generated predictions on an independent subset of MB patients (Fig 3A), using five-fold cross-validation. The T-GAN-D again separated patients into low and high-risk categories with high statistical significance (Figs 3B and S3A). The CNN tested with the same data performed similarly well, either when trained with real patient data (Figs 3C and S3B) or with an augmented training set containing an additional n × 2 synthetic samples (S4 and S6 Figs). The T-GAN-D trained on the merged and reduced dataset also showed improved accuracy when compared to all settings where both a CNN or the GAN were trained on the full or reduced MB dataset alone (S1–S3 Tables). Therefore, in our setting, rescaling and converting transcriptome profiles into images was sufficient to successfully merge the two cohorts without the need for further preprocessing steps and allowed to stratify patients into high and low risk classes.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Introducing the independent TCGA cohort improves MB patient classification.

(A) Schematic representing the training strategy: rescaled data from the entire TCGA cohort were merged with 4/5 of the MB cohort to train the T-GAN-D, which was subsequently used to predict the risk class of the remaining 1/5 of MB patients. The process was iterated 5 times. (B) Kaplan-Meier curves based on the pooled predictions of the T-GAN-D trained on both cohorts. (C) Kaplan-Meier curves separating low vs. high risk patients predicted with the CNN that was trained after merging the MB and the TCGA cohorts. The area between the curves (ABC) between Low risk (blue dashed line) and Predicted low risk (solid blue line), Predicted low risk and Predicted high risk (solid red line), Predicted high risk and high risk groups (dashed red line) are shown top to bottom in B and C.

https://doi.org/10.1371/journal.pcbi.1011035.g003

The T-GAN-D outperforms classical outcome predictors and accurately stratifies early stage patients into risk categories

We next compared the performance of CNN and GAN based classifications to other established clinical markers in breast cancer. These included a scoring system based on a multi-transcript signature (Risk-of-recurrence—proliferation, [ROR-P]), estrogen receptor status (ER), human epidermal growth factor receptor 2 status (HER2), and progesterone receptor status (PR). Likewise, tumor staging was included, yet was available for only 911 out of 1248 patients of the MB cohort. The hazard ratios (HR) obtained from a univariate analysis were comparable for ROR-P, HER2 or tumor staging as classifiers, and similar HRs were also obtained for the CNN and T-GAN-D classifiers developed from only the MB transcriptome dataset (Fig 4A). Interestingly, the T-GAN-D classifier resulting from the merged cohort data returned a mean HR>2.0 (+/- 0.4), thereby surpassing all other common markers (Fig 4A) and could compete with classical ML algorithms (S5 and S6 Figs and S4 Table). This feature was even more pronounced in a multivariate analysis including ER, HER2 and PR biomarkers (Fig 4B). When reducing the MB cohort to those patients for which staging information was available, HRs based on staging and T-GAN-D were comparable (Fig 4C). To test whether both classifiers might be redundant, we performed a T-GAN-D based survival analysis within the tumor stage I and stage II subcohorts, which dominate the MB dataset. T-GAN-D based classification allowed separating high and low risk patients within both tumor stages (Fig 4D and 4E), indicating non-redundancy of the T-GAN-D classification to tumor staging information.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. The T-GAN-D outperforms classical biomarkers after merging the MB and TCGA cohorts and significantly stratifies early stage MB patients.

(A) Comparison of the hazard ratios (Cox model, univariate) of a multi-transcript signature (ROR-P) and established prognostic biomarkers (ER, HER2, PR) vs. the CNN and the T-GAN-D before and after cohort merging. (B) Multivariate Cox hazard ratio of the T-GAN-D compared to ROR-P and receptor status and (C) disease stage. (D) Kaplan -Meier curves of Stage I and (E) Stage II patients stratified by the T-GAN-D into low and high risk categories.

https://doi.org/10.1371/journal.pcbi.1011035.g004

Taken together, these results show enhanced performance of the discriminator network that was exposed to additional synthetic samples during the training process of the GAN. In our use case scenario, the T-GAN-D surpasses individual classical biomarkers and performs well when prognosticating early stage breast cancer cases.

The T-GAN-D stratifies TCGA patients despite these being scarcely represented

After observing that introducing TCGA patients into the training set of the T-GAN-D did not degrade, but improved the stratification of MB patients, we tested the performance of the classifier on the smaller TCGA dataset. To do this, we retrained the GAN using the entire MB data plus a fraction of the TCGA data from the merged dataset and used the discriminator to generate predictions on an independent subset of TCGA patients (Fig 5A), using five-fold cross-validation. The T-GAN-D correctly predicted 78% of the cases (Figs 5B and S7A and S5 Table). In contrast, when trained on the MB dataset alone, the T-GAN-D was not able to separate high and low risk patients (Figs 5C and S7B), achieving an overall accuracy of only 43% (S5 Table). Therefore, the addition to the training set of a comparably small number of TCGA patients (n = 129) to the larger MB cohort (n = 1244) was sufficient to drastically improve the performance of the T-GAN-D predicting TCGA patient outcome. This demonstrates that even if the training set is largely dominated by patients belonging to one cohort, the introduction of a limited number of samples of a second, differently balanced dataset appears sufficient to possibly capture relevant patterns that contribute to achieving improved prognostic performance.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. The T-GAN-D stratifies TCGA patients despite these being scarcely represented in the merged training set.

(A) Schematic representing the training strategy: rescaled data from the entire MB cohort were merged with 4/5 of the TCGA cohort to train the T-GAN-D, which was subsequently used to predict the risk class of the remaining 1/5 of TCGA patients. The process was iterated 5 times. (B) Stratification of the TCGA patients by T-GAN-D trained on the merged dataset and (C) the MB dataset alone. Kaplan-Meier curves were generated pooling the predictions of all iterations of the 5-fold CV. The area between the curves (ABC) between Low risk (blue dashed line) and Predicted low risk (solid blue line), Predicted low risk and Predicted high risk (solid red line), Predicted high risk and High risk groups (dashed red line) are shown top to bottom in B and C.

https://doi.org/10.1371/journal.pcbi.1011035.g005