Association between cancer mutations from Cancer Hotspots and LP/P classification as germline variants

Putative driver mutations from Cancer Hotspots were extracted, annotated, and filtered to obtain a list of 2,447 missense mutations (“CH mutations”) distributed across 216 genes (Fig 1C). Of these 216 genes, 41% are proto-oncogenes, 36% are tumour suppressor genes, and 15% can have either role, as determined by the Cancer Gene Census (S1A Fig) [25]. We presumed that cancer driver missense mutations in proto-oncogenes and tumour suppressor genes have gain of function and loss of function mechanisms, respectively. The Mendelian disease associations in the Online Mendelian Inheritance in Man (OMIM) database [26] for these genes revealed that 20% are associated with hereditary cancer predisposition syndromes (Table A in S2 Text). Among the 216 genes, 154 had known modes of inheritance for cancer and an associated Mendelian disease reported in OMIM [26]. Of these 154 genes, 107 (69%) had a Mendelian disease mechanism that was concordant with the cancer mechanism, 26 (17%) were discordant, and 21 (14%) were semi-concordant, meaning the gene could function as both a proto-oncogene and a tumor suppressor, or had Mendelian diseases with variants exhibiting both gain of function and loss of function mechanisms (Table B in S2 Text). Although Cancer Hotspots infers cancer driver status of a mutation from probabilistic arguments (statistical enrichment), we found that the functional impact was experimentally tested for 990 of these mutations with the majority (943/990, 95%) confirmed to result in gain or loss of protein function (S1 Text and S2 Fig).

Overall, 691 missense mutations in 84 genes from Cancer Hotspots had been classified with respect to germline pathogenicity in ClinVar: 426 (61.6%) as LP/P, 261 (37.8%) as VUS, and 4 (0.6%) as LB/B (Fig 1C). The median number of variants observed for each gene was 2 (interquartile range = 4). As expected, all variants were rare (gnomAD allele frequency < 0.001) except for three out of four that were classified as LB/B. Of these 84 genes, 50% are proto-oncogenes, 37% are tumour suppressor genes, and 10% can have either role, as determined by the Cancer Gene Census (S1B Fig). Germline variants overlapping with cancer (driver) mutations may provide insights into their mechanisms, such as loss of function in tumor suppressor genes or gain of function in proto-oncogenes and provide functional context for Mendelian diseases. The disease associations in OMIM for these genes also revealed that 38% were hereditary cancer predisposition syndromes (e.g., VHL associated with von Hippel-Lindau syndrome) and 62% were not known to include cancer as a predominant feature (e.g., FGFR3 associated with Achondroplasia) [26]. In both groups, most associated conditions had autosomal dominant inheritance (88% and 77%, respectively). A significant difference was observed in the proportion of LP/P, VUS, and LB/B variants between these two gene groups (256 LP/P, 231 VUS, 1 LB/B versus 170 LP/P, 30 VUS, 3 LB/B, respectively), with an LP/P classification more likely for variants in genes not associated with hereditary cancer predisposition syndromes (p < 2.2e-16) (Table A in S2 Text).

The odds ratio for these 691 variants having a LP/P classification in ClinVar was 107.6 (95% confidence interval (CI): 40.1–288.4, p < 0.0001), when comparing only LP/P and LB/B classifications with all other germline missense variants with ClinVar entries in the 216 genes (n = 5,474) (S3 Fig and Table C in S2 Text). Even if all VUS were considered as LB/B variants, the odds ratio was 28.3 (95% CI: 24.2–33.1, p < 0.001) compared with all other variants in ClinVar (n = 50,655) (S3 Fig and Table C in S2 Text). In an even more extreme scenario of considering all VUS and CIP variants as LB/B, the odds ratio was 21.0 (95% CI: 18.2–24.2, p < 0.001) (n = 53,593) (S3 Fig and Table C in S2 Text). If these variants were restricted to the 107 genes with Mendelian disease mechanism that was concordant with the cancer mechanism, 337 cancer mutations would overlap with germline missense variants in ClinVar (238 LP/P, 98 VUS, 1 LB/B). The odds ratio for an LP/P classification in ClinVar would increase to 46.2 (95% confidence interval: 36.4–58.6, p < 0.001), compared to all other germline missense variants in the same 107 genes. However, the odds ratio for LP/P classification for the “discordant” and “semi-concordant” mechanisms was still 12.5 (95% confidence interval: 9.9–15.7, p < 0.001). The most conservatively estimated positive likelihood ratio of 18.3 still exceeded “moderate evidence” thresholds described previously (i.e., 4.33 and 5.79) (S1 Text and Table D in S2 Text) [27,28]. The potential impact of an additional moderate evidence criterion for pathogenicity applied to the 261 CH mutations that overlap with germline VUS in ClinVar is shown in S4 Fig, revealing 66 (27%) of the VUS could be hypothetically upgraded to LP.

For the remaining CH mutations that did not overlap with germline variants in ClinVar (n = 1,756), we explored the degree to which in silico scores used for germline variant adjudication supported “pathogenicity”. We grouped these CH mutations by REVEL scores using the ClinGen-proposed PP3/BP4 score thresholds (Fig 1C) [28]. Over half (58.8%; 1,032) had REVEL scores indicating at least PP3-level evidence (i.e., evidence in favour of pathogenicity), while only 9.6% (168) had at least BP4-level evidence (Figs 1C and S5A). Findings were similar using AlphaMissense (S5B Fig) [29]. For these CH mutations that are absent from ClinVar, the in silico score profiles resemble the ClinVar LP/P germline missense variants in the same genes more than the set of LB/B variants or VUS (S5 Fig).

Through collaborations with GEL, MSSNG, C4R, and GeneDx (see Methods), we searched GWS datasets from approximately 1.5 million participants (probands and affected or unaffected family members) and identified additional instances of germline variants overlapping with CH mutations (Table E in S2 Text). Across the four datasets, we found 302 unique overlapping germline variants. Of these, 194 were already classified and present in ClinVar (140 LP/P, 1 LB/B, 53 VUS) and 108 were absent in ClinVar. Out of these 108 variants, 43 had been previously assessed and classified in accordance with ACMG/AMP variant interpretation guidelines by our collaborators. Among these variants, 30 were classified as LP/P, 12 as VUS, and 1 conflicting (LP and VUS by different groups). The classifications of the remaining 65 variants (79% found in probands) were uncertain due to limited phenotype information.

Cancer Hotspots database includes most highly recurrent cancer mutations in COSMIC

We retrieved 231,377 somatic missense mutations by filtering the Cancer Census Genes data from COSMIC (S6 Fig). With the results of the tumour sample count analysis using overlapping CH mutations and ClinVar germline variants (S1 Text and S7 Fig), we stringently filtered for COSMIC mutations that were observed in >25 tumour samples and absent from Cancer Hotspots, resulting in 125 missense mutations across 63 genes (S6 Fig). This approach, using Cancer Hotspots as a benchmark, aimed to identify recurrent (putative) driver mutations in COSMIC, a more heterogeneous database with both driver and passenger mutations. Of these genes, 31 are new additions to the list of genes from Cancer Hotspots and 11 are associated with rare Mendelian diseases as reported in OMIM [26]. However, only 12 of these mutations overlapped with germline variants in ClinVar. Among them, 2 (16.7%) were LP/P, 8 (66.7%) VUS/CIP and 2 (16.7%) were LB/B (S6 Fig). Only 2 of these 12 overlapping variants were found in the “new” 31 cancer genes discovered through COSMIC. While we identified 125 additional missense mutations in COSMIC, only a small fraction of these overlapped with germline variants in ClinVar. Thus, despite being smaller and less frequently updated than COSMIC, Cancer Hotspots effectively captures most putative cancer driver missense mutations relevant to our research question.

Robust predicted probabilities of pathogenicity generated by supervised learning models

We used the training dataset to develop two types of supervised learning models with the goal to accurately predict the pathogenicity of germline variants in our test dataset. The training dataset fit the LRM with a McFadden’s pseudo-R² value of 0.50 (i.e., higher than the 0.20–0.40 range that indicates a good model fit [30] and generated predicted probabilities of pathogenicity for all variants in the training dataset. The predicted probabilities were significantly higher for all germline LP/P variants compared with LB/B/VUS variants (U = 1655893, n_LB/B/VUS = 11,644, n_LP/P = 2,095, p < 0.0001) and for germline variants that are present in the Cancer Hotspots database compared with those that are absent (U = 32029, n_Absent = 13,316, n_Present = 423, p < 0.0001) (Fig 2A and 2B). We trained a second supervised learning model, an RFM, since it is gene-independent and can be broadly applied to variants beyond the 66 gene categories in the LRM. The RFM achieved an out-of-bag (OOB) error estimate of 10.8% for predicting outcomes. The RFM generated probability scores of pathogenicity and, similar to the LRM, these were significantly higher for all germline LP/P variants compared with LB/B/VUS variants, as well as for germline variants that overlap with CH mutations compared to those without overlap (U = 6109589, n_LB/B/VUS = 11,644, n_LP/P = 2,095, p < 0.0001) (Fig 2C and 2D). To gain a comprehensive understanding of the overall impact of each independent variable on the data, exploratory analyses were conducted on the ClinVar dataset (before filtering) (S1 Text and S7–S10 Figs). The analyses show variability in the number of variants across genes (S8 Fig), distinct tumour sample count thresholds between LP/P and LB/B/VUS variants (S7 Fig) and indicated that the model fit was not primarily driven by the conservation scores (S9 and S10 Figs).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Fit of training dataset using supervised learning models.

(A) Plot of predicted probabilities of pathogenicity for all likely benign/benign/variant of uncertain significance (LB/B/VUS) and likely pathogenic/pathogenic (LP/P) in the training dataset assigned by the logistic regression model. Mann-Whitney U test: U = 1655893, n_LB/B/VUS = 11,644, n_LP/P = 2,095. (B) Comparison of predicted probabilities for germline variants with absence or presence of overlap with cancer mutations. Mann-Whitney U test: U = 32029, n_Absent = 13,316, n_Present = 423. (C) Plot of probability scores of pathogenicity for LB/B/VUS and LP/P in the training dataset assigned by the random forest model. Mann-Whitney U test: U = 6109589, n_LB/B/VUS = 11,644, n_LP/P = 2,095. (D) Comparison of probability scores for germline variants with absence or presence of overlap with cancer mutations. Mann-Whitney U test: U = 12913, n_Absent = 13,316, n_Present = 423. Created with GraphPad Prism.

https://doi.org/10.1371/journal.pgen.1011540.g002

RFM outperformed LRM in correctly predicting pathogenicity of germline missense variants overlapping with cancer mutations

Using the test dataset (n = 335), distinct from training dataset variants, we calculated the area under precision-recall curve (AUPRC) values for the LRM and RFM as 0.847 and 0.829, respectively (Fig 3A). We also calculated the area under the receiver-operating characteristic curve (AUROC) as 0.821 for the LRM and 0.774 for the RFM (S11A Fig). The higher AUROC for the LRM indicates better ability to discriminate between LP/P and LB/B/VUS variants compared to the RFM. Precision-recall curves guided the selection of optimal classification thresholds, with an emphasis on minimizing false positives while maximizing AUPRCs. The LRM had an optimal threshold of 0.74 (F1 score = 0.690) (S12A Fig). The RFM had an optimal threshold of 0.39 (F1 score = 0.783) (S12B Fig), with the higher F1 score compared with the LRM indicating superior performance in correctly predicting the pathogenicity of test dataset variants.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Evaluation of supervised learning models.

Precision-recall curve comparing the performance of the logistic regression model (blue) and the random forest model (purple) using the (A) test dataset and (B) cross-validation set. The models’ performance was evaluated using k-fold cross-validation, with k = 8 for logistic regression and k = 10 for random forest. AUC, area under the curve.

https://doi.org/10.1371/journal.pgen.1011540.g003

We compared the performance of the LRM and RFM pathogenicity scores against the scores of other in silico prediction tools by plotting precision-recall curves and comparing the calculated AUPRCs (S13A Fig). The LRM and RFM outperformed the first-generation tools [31] SIFT and PolyPhen-2, which had AUPRCs of 0.821 and 0.827, respectively (S13B Fig). Second- (REVEL, CADD, VARITY, VEST4) and third-generation (AlphaMissense, PrimateAI, MutPred2) [31] tools demonstrated a stronger performance in classifying the test dataset variants, with AUPRCs ranging from 0.881 to 0.963 (S13C and S13D Fig). REVEL, VARITY, and AlphaMissense were the top-performing tools, respectively. Given the smaller size of the test dataset compared with the training dataset, cross-validation techniques were also used to confirm the LRM and RFM’s reliability in estimating performance (Figs 3B and S11B). The RFM consistently outperformed the LRM in terms of AUPRC, exhibiting a higher value than was observed with the test dataset alone (0.940 versus 0.738 AUC). Although the LRM had a higher AUROC (0.928) compared to the RFM (0.739), AUROC reflects overall discriminative ability across all thresholds, whereas AUPRC and F1 scores are more relevant for assessing performance in detecting positive cases. We used the RFM and the optimal threshold value of 0.39 to predict pathogenicity of the 65 variants with unknown classification identified through our collaborations with MSSNG, GEL, C4R, and GeneDx. Of these 65 variants, the RFM predicted 92% to be LP/P and 8% as LB/B. The average probability score of pathogenicity for the predicted LP/P variants was 0.93 and 80% were in probands.

Ethics statement

This secondary use data study was approved by the Research Ethics Board at the Hospital for Sick Children. The de-identified data from GeneDx was assessed in accordance with an IRB-approved protocol (WIRB #20171030).

Extracting cancer mutation data from cancer Hotspots

We obtained cancer mutation data for 3,122 single nucleotide variants (SNVs) from the Cancer Hotspots [22,23] database (www.cancerhotspots.org), representing a set of true cancer driver mutations. This database consists of mutational hotspots identified in large scale cancer genomics data, defined as single amino acid positions in protein-coding genes that are mutated more frequently than would be expected in the absence of selection [13,23]. This method assigns a statistical significance to the recurrence of mutation at a given amino acid and is corrected for background mutational rate of the position, gene, and sample both within and across cancer types in the affected cohort [22,23]. Somatic mutational hotspots are therefore not common germline benign variants in a population [13,22,23]. A Python script was developed to extract genomic coordinates in GRCh37, reference and alternate alleles, and tumour sample counts for each mutation. Only missense mutations (n = 2,576) were used for our analyses. We annotated the cancer missense mutations using ANNOVAR and a custom pipeline [2] developed by The Centre for Applied Genomics (Toronto, Canada). ClinVar annotations (date accessed: Jan 2022) were used to identify clinical classifications of those germline variants that are also cancer mutations in Cancer Hotspots. We conservatively excluded any mutations with corresponding germline variants with “conflicting interpretations of pathogenicity” (CIP) or considered a “risk factor” for disease (n = 129). The remaining 2,447 recurrent missense mutations (n = 216 total genes) from Cancer Hotspots are hereafter referred to as the “CH mutations”.

Comparing cancer mutations with germline variants

Separately, we extracted from ClinVar (date accessed: Jan 2022) all missense variants in the 216 genes from the list of CH mutations (n = 51,346 SNVs) (S3 Fig). We selected missense variants with a “germline” allele origin, i.e., excluding those labeled as “somatic” or “unknown”. These variants were then grouped into three categories based on their ACMG classification in ClinVar: “likely pathogenic” or “pathogenic” (LP/P) (n = 3,149), “likely benign” or “benign” (LB/B) (n = 2,755), and “variant of uncertain significance” (VUS) (n = 45,442). We annotated these variants using ANNOVAR to include REVEL [42], phyloP [43] (20way mammalian and 7way vertebrate), and phastCons [44] (20way mammalian and 7way vertebrate) scores. For each variant, we noted the presence or absence of an overlap with a CH mutation. These variants are hereafter referred to as the “ClinVar dataset” and were used to calculate the odds ratios of a germline variant that overlaps with a CH mutation having an LP/P classification. This data was also used to apply mathematical framework described by Tavtigian et al. to define ACMG/AMP evidence strength for the use of cancer mutational hotspot data for germline variant interpretation [27].

Identifying overlap with cancer mutations in other genomic databases

We queried the CH mutations in four controlled-access GWS databases, in collaboration with MSSNG [45], Genomics England [46] (GEL), Care4Rare [47] (C4R), and GeneDx [9,48,49], to identify matching germline missense variants (at the nucleotide level).

The MSSNG database represents a cohort of autistic individuals / individuals with autism and their family members. All germline missense variants in this database were extracted and converted to GRCh37 using LiftOver. Germline variants in MSSNG, and CH mutations, were imported to R version 4.1.0 (R Foundation for Statistical Computing) to identify overlapping variants by genomic coordinate, reference allele, and alternate allele. The GEL, C4R, and GeneDx databases represent phenotypically heterogeneous cohorts of individuals with suspected rare genetic diseases and their family members. In the GEL Research Environment, a bash shell script was used to extract variants from variant call format (VCF) files by genomic coordinates. The CH mutations were queried against germline variants in the VCF files of all participants in the Rare Disease program of GEL using this script. The participant IDs for each CH mutation that overlapped with a germline variant in GEL were used to retrieve phenotype data along with their classifications using the Labkey platform. In collaboration with C4R and GeneDx, the CH mutations were sent to the respective study teams and queried within their databases. Results of overlapping variants and participant IDs were returned. Variant classification and phenotype data from C4R was explored by searching the Genomics4RareDisease (G4RD) database with participant IDs [50].

Identifying cancer mutations from other cancer databases and comparing with germline variants

We downloaded approximately 1.1 million coding mutations from the COSMIC database [51] listed in the Cancer Gene Census [25] and filtered for confirmed somatic missense mutations (n = 231, 477). To align with the stringent criteria used in the Cancer Hotspots database, we further filtered based on the presence of mutations in COSMIC across a defined number of tumor samples. This step ensured the retention of only those mutations observed across a substantial number of tumors, indicative of potential driver mutations as defined in Cancer Hotspots. For this filtering process, we used tumor sample counts of CH mutations that overlap with germline variants in ClinVar (S1 Text). Plotting these values by ClinVar classification groups (LP/P and LB/B/VUS), we generated receiver operating characteristic (ROC) curves to determine the optimal tumor sample count cut-off for distinguishing between LP/P and LB/B/VUS variants. The identified optimal count was then used to filter the COSMIC mutations. We then conducted further filtered to identify “new” mutations in COSMIC, i.e., those absent in Cancer Hotspots, and compared these mutations with germline variants in ClinVar, to identify additional overlapping variants.

Training dataset used for supervised learning models

We developed supervised learning models to predict pathogenicity of unclassified germline variants, based on a set of variants with known classifications in ClinVar. To construct the training variant set, we used the ClinVar dataset including n = 51,346 SNVs in the 216 genes from the list of CH mutations. Different nucleotide variants resulting in the same amino acid change were grouped together. VUS with REVEL scores >0.29 were excluded from the training dataset. This cut-off is the upper-most bound for BP4 evidence level for REVEL scores [28]. The remaining VUS were included and treated as LB/B variants (Fig 4; see below regarding weighting), to address class imbalance arising from fewer LB/B versus LP/P variants in the dataset. Variants were then restricted to a set of 66 genes, determined by the updated list of 428 CH mutations overlapping with germline variants (Fig 4). The resulting training dataset comprises 13,881 variants.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. Training dataset for supervised learning models.

The training dataset is comprised of 13,881 germline missense variants from ClinVar (green), including 691 overlapping with cancer mutations (blue). Different single nucleotide changes causing the same amino acid change were grouped together accounting for the difference in the overlap shown in Fig 1. Variants of uncertain significance (VUS) with REVEL scores ≤ 0.290 were included in the dataset and treated as likely benign/benign (LB/B) variants (see text for justification). LP/P, Likely pathogenic/Pathogenic.

https://doi.org/10.1371/journal.pgen.1011540.g004

Developing supervised learning models

Two types of supervised learning models were fit to the training dataset in R: a logistic regression model (LRM) and a random forest model (RFM). Pathogenicity status (LB/B, LP/P) was used as the dependent variable and the following were used as independent variables: 1) overlap with a cancer missense mutation from Cancer Hotspots (2 categories: present = 1, absent = 0), 2) the protein-coding gene associated with a variant (with 66 categories representing each gene), 3) the number of tumour samples with a specific amino acid change at a residue position from Cancer Hotspots, 4) the number of tumour samples with a mutated residue from Cancer Hotspots, 5 & 6) the phyloP conservation scores [43] (20way mammalian and 7way vertebrate), and 7 & 8) the phastCons conservation scores [44] (20way mammalian and 7way vertebrate).

The ’stats’ R package was used to fit the LRM. REVEL scores for the included VUS (all < = 0.29) were used as prior weights (weight = 1—REVEL score) compared to true LB/B variants (weight = 1). The predicted probabilities and standard performance metrics including Akaike Information Criterion (AIC) and McFadden’s pseudo-R² were used to assess the fit of the model. The same training dataset was used for the RFM using the ’randomForest’ package in R. However, the gene variable was excluded due to a categorical variable limit of 32 levels. Hyperparameters for the RFM, including the number of classification trees (350) and the number of independent variables randomly selected for each split (4), were selected based on plateau of the out-of-bag (OOB) error rate using the training dataset.

Evaluating supervised learning models with test dataset

Both LRM and RFM performance was evaluated using a test dataset of 335 germline missense variants that were absent from the training dataset. In the test dataset, 53.4% were LP/P (n = 179) and 46.6% are LB/B/VUS (n = 156), with 19.4% (n = 65) variants present in Cancer Hotspots and 80.6% (n = 270) variants absent. These variants were obtained from new ClinVar submissions from Feb 2022 to Aug 2022 (n = 185), the Leiden Open Variation Database (LOVD) [52] (n = 35), G4RD database [50] (n = 1), GEL database [53] (n = 93), SickKids Cancer Sequencing (KiCS) dataset [54] (n = 2), and from manual review of literature pertaining to the genes of interest that was published from 2021–2022 (n = 19). The test dataset variants impact genes that are represented in the training dataset. We used the predicted classifications of each model across all possible classification thresholds to plot precision-recall curves and calculate the area under the curve (AUPRC). The highest performing model and optimal threshold were used to assess the pathogenicity of an additional set of variants with unknown classification identified in other genomic databases through collaborations. The variants in the test dataset were annotated using scores from other in silico prediction tools, including SIFT [55], PolyPhen-2 [56], REVEL [42], CADD [57], VARITY [58], AlphaMissense [29], PrimateAI [10], VEST4 [59], and MutPred2 [60]. Some tools were selected because they are commonly used for variant interpretation in the diagnostic laboratory, are referenced in ACMG/AMP guidelines [4], and/or are incorporated into annotation tools like ANNOVAR. The remaining tools (e.g., AlphaMissense) were selected because of their strong potential to be incorporated into clinical interpretation workflows in the future. We also plotted precision-recall curves using these scores to calculate the AUPRCs and compared them with the LRM and RFM.

Evaluating supervised learning models with cross-validation

Cross-validation was conducted using the ’caret’ package in R, with the ’createFolds’ function employed to generate the folds for model training and evaluation. The training dataset was divided into k folds, where the model was trained on k-1 fold and tested on the remaining one. The training dataset was divided into 8 and 10 folds for the LRM and RFM, respectively. The F1 score and AUPRC, using a threshold of 0.5, was calculated for each fold, and averaged over the k folds to obtain an estimate of each model’s generalization ability. Cross-validation for the RFM used the same hyperparameters (350 classification trees and 4 independent variables per split) as the RFM trained without cross-validation for each fold.

Statistical methods

Standard descriptive statistics, odds ratios, and Mann-Whitney U tests were performed using R and GraphPad Prism 9 with two-tailed statistical significance set at p < 0.05.