Clinical Validation of Targeted Next Generation Sequencing for Colon and Lung Cancers

Objective Recently, Next Generation Sequencing (NGS) has begun to supplant other technologies for gene mutation testing that is now required for targeted therapies. However, transfer of NGS technology to clinical daily practice requires validation. Methods We validated the Ion Torrent AmpliSeq Colon and Lung cancer panel interrogating 1850 hotspots in 22 genes using the Ion Torrent Personal Genome Machine. First, we used commercial reference standards that carry mutations at defined allelic frequency (AF). Then, 51 colorectal adenocarcinomas (CRC) and 39 non small cell lung carcinomas (NSCLC) were retrospectively analyzed. Results Sensitivity and accuracy for detecting variants at an AF >4% was 100% for commercial reference standards. Among the 90 cases, 89 (98.9%) were successfully sequenced. Among the 86 samples for which NGS and the reference test were both informative, 83 showed concordant results between NGS and the reference test; i.e. KRAS and BRAF for CRC and EGFR for NSCLC, with the 3 discordant cases each characterized by an AF <10%. Conclusions Overall, the AmpliSeq colon/lung cancer panel was specific and sensitive for mutation analysis of gene panels and can be incorporated into clinical daily practice.

Introduction required. The ethical committee has thus waived the need for written informed consent from the participant.

Samples selection
Tumor samples from 90 patients were retrospectively analyzed, including 51 colorectal adenocarcinomas (CRC) and 39 non small cell lung carcinomas (NSCLC including 37 adenocarcinomas and 2 squamous carcinomas). The mutational status of KRAS and BRAF in CRC and of EGFR in NSCLC had been assessed previously in the context of daily practice. The primary sample types were either surgical resections (n = 57, 44 CRC and 13 NSCLC), biopsies (n = 23, 7 CRC and 16 NSCLC) or cell blocks (n = 10, all NSCLC). In addition, we used 12 non neoplastic samples (6 lungs and 6 colons) and 5 commercial FFPE reference standards (Horizon Diagnostics, Cambridge, UK) carrying mutation in NRAS, KRAS, AKT and EGFR at 50% allelic frequency (AF) and 1 FFPE multiplex reference standard (Horizon Diagnostics, Cambridge, UK) carrying 11 different mutations at various defined AF (from 0.9 to 24.4%).

DNA extraction
DNA was extracted from FFPE tumor samples using the QIAamp FFPE tissue kit (Qiagen, Antwerp, Belgium). Briefly, unstained 10 μm paraffin sections were cut and incubated at 37°C in a drying oven overnight. The paraffin was removed by incubating the slides in 2 successive baths of xylene and the tumor tissue was manually macrodissected, scraped off the slide with a scalpel and transferred into a 1.5ml tube. DNA was then extracted according to the manufacturer's instructions. The H&E stained slide from the same block, previously reviewed by a pathologist who circled the tumor area and evaluated the tumor percentage, was used as a guide for the macrodissection. The percentage of tumor cells of the samples ranged from 5 to 90%. The DNA obtained was quantified using the Qubit 1 fluorometer in combination with the Qubit 1 dsDNA HS assay kit (Life Technologies, Gent, Belgium).

Detection of KRAS, BRAF and EGFR mutations
Detection of KRAS, BRAF and EGFR mutations were performed in the context of clinical daily practice in an ISO15189-certified laboratory by quantitative PCR. These methods are described in S1 File. The sensitivity of these assays is varied between 3 and 20% of mutant DNA for the KRAS testing, 10% of mutant DNA for BRAF testing, 0.5% of mutant DNA for EGFR p.L858R testing, 1% of mutant DNA for EGFR exon 19 deletion and 5% of mutant DNA for EGFR p. T790M testing, Droplet digital PCR Some mutations detected by NGS were validated by droplet digital PCR (ddPCR), as detailed in S1 File.

Data Analysis
The raw data were analyzed using the torrent suite software v3.6.2 (Life technologies). The coverage analysis was performed using the coverage analysis plug-in v3. 6. Cases for which the number of mapped reads was <100000 and/or the average base coverage was <500x were considered as non informative. Mutations were detected using the Variant Caller plug-in v3.6 with low stringency settings (Life Technologies). In the variant list obtained, each mutation was verified in the Integrative genome viewer (IGV) from the Broad Institute (http://www. broadinstitute.org/igv/) [19]. Only mutations reported in the COSMIC (Sanger Institute Catalogue of Somatic Mutations in Cancer) database (http://www.sanger.ac.uk/cosmic) were taken into account and silent or intronic mutations were not reported.

Statistical analyses
The non-parametric Mann-Whitney and Kruskal-Wallis tests were used to compare two, or multiple, independent groups of numerical data, respectively. If the Kruskal-Wallis test was significant, post-hoc tests were applied using either the standard Dunn procedure to compare all group pairs or its adaptation to compare each experimental condition to the control, avoiding multiple comparison effects (as detailed in Zar [20]).
All statistical analyses were performed using Statistica (Statsoft, Tulsa, OK, USA) and pvalues < 0.05 were considered significant.

NGS panel validation
The performance of the AmpliSeq Colon and Lung Cancer panel was first evaluated using 12 non neoplastic tissues (6 lungs and 6 colons) and 6 commercial FFPE reference standards (5 reference standards with one mutation at 50% allelic frequency and one multiplex reference standard carrying 11 different mutations at various defined allelic frequencies, varying from 0.9 to 24.4%).
No mutation was detected in the 12 non neoplastic tissues. The 5 mutations present in the 5 reference standards at 50% allelic frequency were all correctly detected by NGS with the Ampli-Seq Colon and Lung Cancer panel (Table 1). Among the 11 mutations present in the multiplex reference standard, all mutations with AF >3% were correctly detected by NGS with the exception of the KIT mutation because this gene is not included in the 22 genes of the panel. For the 3 mutations with AF <3%, only one (EGFR deletion in exon 19, AF = 2.0%) was detected by the Variant Caller whereas the two others (EGFR p.L858R and p.T790M with AF = 2.7% and 0.9% respectively) were not. By IGV inspection, we found that these variants were present but with low AF (25/1633 reads (1.5%) and 7/1613 reads (0.4%), respectively). Additional mutations in CTNNB1, BRAF, PIK3CA and EGFR were detected by the Variant Caller in the reference standards ( Table 1). The KRAS and NRAS standards are generated from the SW48 cell line which is reported to carry CTNNB1 p.S33Y and EGFR p.G719S mutations in the COSMIC database (http://www.sanger.ac.uk/cosmic); the EGFR standards are generated from the RKO cell line which is reported to carry BRAF p.V600E and PIK3CA p.H1047R mutations. Finally, the multiplex reference standard is generated from the RKO, SW48 and HCT16 cell lines, which explains the detection of the CTNNB1 p.S33Y mutation in this control. The precision (reproducibility and repeatability) was also evaluated using 2 FFPE tumour samples and the multiplex reference standard. The samples were analysed 5 times (5 library productions starting from the same DNA extract) in three different experiments ( Table 2). All mutations with an AF >4% were consistently detected. However, mutations with an AF <3% were detected by the Variant Caller in one or more, but not all, of the five replicates. By IGV inspection, the TP53 mutations inconsistently detected by the Variant Caller were present only in the replicate for which the mutation was detected by the Variant Caller, but not for the other replicates (S1 Table). For the reference standard, the 3 EGFR variants were detected by IGV inspection (but with a variant coverage < 30x for the majority of the replicates) although these were inconsistently detected by the Variant Caller (S1 Table).
Moreover, some mutations were verified by ddPCR ( Table 2). The p.H1047Q PIK3CA mutation, consistently detected by NGS with a mean AF of 10.8%, was also detected by ddPCR with an AF of 9.1%. In contrast, the p.R181C and the p.H168Y TP53 mutations inconsistently detected by NGS were not detected by ddPCR. Given the facts that mutations detected with an AF < 3% were not validated by ddPCR (for TP53 mutations) or inconsistently detected (EGFR mutations in the reference standard), and that the KRAS mutation with an expected AF of 5% was consistently detected with an AF varying from 4.6 to 5.9%, we selected a 4% AF threshold for mutation reporting. This threshold is consistent with the data reported in the literature for this NGS platform [9,16,21].

Sequencing performances
A set of 90 FFPE samples, including 51 CRC and 39 NSCLC, was sequenced by NGS. Sequencing performance was assessed from the number and distribution of reads across the targeted regions. Among the 90 sequenced cases, 89 (98.9%) were successful (number of mapped reads >100000 or average base coverage >500x). The unsuccesfull case was considered non informative because of a number of reads <100 000 (40.072 reads) and average base coverage <500x (1.2X). Among the 89 successfully sequenced cases, one case was considered suboptimal (number of reads: 69.564 and average base coverage: 676x), however the quality of the sequencing was considered good enough for further analysis. All the other cases (n = 88, 97.8%) had a number of reads >100.000 and an average base coverage >1000X. The average number of reads per samples was 232.832 and the average base coverage depth was 2.296. On average 91.6% of the amplicons had a coverage depth of more than 500x (Table 3). There was no significant difference between CRC and NSCLC in terms of number of reads (p = 0.45), base coverage depth (p = 0.42) and percentage of amplicons with a coverage depth higher than 500x (p = 0.32) (Mann-Whitney test). For 12 cases, the amount of DNA obtained after extraction was too low to reach the required 10ng for targeted sequencing (DNA concentration ranging from 0.1 to 1.5ng/μl). The sequencing was successful for the 12 cases and no statistical difference was observed in terms of number of reads (p = 0.45), base coverage depth (p = 0.42) and in terms of percentage of amplicons with a coverage depth of more than 500x (p = 0.32) between samples with less than the required 10ng of DNA and samples with enough DNA (Mann-Whitney test, Table 3). This suggests that successful sequencing can be obtained from as little as 1 ng of DNA. We then considered the influence of different primary sample types on the sequencing performance. No statistical difference was observed in terms of number of reads and base coverage depth between surgical resections, biopsies and cell blocks (Kruskal-Wallis test, Table 3). However, the percentage of amplicons with a coverage depth of more than 500X was significantly higher for cell blocks than for biopsies (Kruskal-Wallis test p = 0.02 and post-hoc test p = 0.02).
The amplicons that showed a coverage below 250X were considered as non informative. This threshold was already used in the literature [22]. Some of the amplicons of the panel repeatedly failed to reach 250X (in more than 10% of the samples). These amplicons are listed in Table 4. The amplicons that repeatedly failed were the same for the different primary sample types (biopsies, cell blocks and surgical resection). For these amplicons, the GC content was significantly higher (average GC content in the 7 amplicons that repeatedly failed was 69.6% against 48% for the other amplicons, p = 0.00005), whereas the length was not significantly different (the average length of the 7 amplicons that repeatedly failed was 108 bp against 112 for the other amplicons).

Comparison with other methods
A set of 90 FFPE samples, including 51 CRC and 39 NSCLC, was sequenced by NGS. The mutational status of KRAS (exon 2) and BRAF (p.V600E) in CRC and EGFR (p.L858R and deletions in exon 19) in NSCLC had been previously assessed by PCR in the context of daily practice.
NSCLC. Sequencing was successful for 38 of the 39 samples tested (97%) whereas the EGFR analysis with the PCR method was successful for only 35 of the 39 samples (90%). 34 samples have successful testing both by NGS and PCR, allowing study of concordance.
Using NGS, mutations in EGFR were detected in 4/38 cases (10.5%). Three out of four EGFR mutations were also detected by PCR. For one sample that was considered as non informative by PCR, a p.L861Q EGFR mutation was detected using NGS (S2 Table). Moreover, a p. L858R EGFR mutation was detected by PCR and by NGS. However, for this sample the Variant Caller detected the mutation at an AF of 1.9%; given our criteria to consider a variant as authentic (see material and methods) we could not validate this variant.
CRC. Sequencing and PCR were successful for all samples.
Using NGS, mutations in KRAS were detected in 30/51 cases (58.8%), whereas KRAS analysis with the PCR method detected only 23 cases of mutations in the KRAS gene (45.1%). Five of the 7 discordant cases were characterized by mutations in codons 59, 61 and 146 (exons 3 and 4) that were not covered by the PCR test. Mutations in codons 61 and 146 were tested and validated by ddPCR (S3 Table) For the 2 remaining cases not detected by PCR, KRAS p.G12V mutation was detected by NGS with an AF of 8 and 9%, respectively. These mutations were also detected by ddPCR with an AF of 12.5 and 9%, respectively. In the 23 concordant cases, AF of KRAS mutations were higher than 20% for 21 cases; only two cases were characterized by an AF of 9 and 10%, respectively.
Mutational profiles of CRC were summarised in Fig 2 and S3 Table. Overall, 45/51 samples were characterized by at least one mutation (88.2%).

Discussion
A major advantage of NGS over traditional mutation detection methods is its ability to screen multiple mutations in multiple genes simultaneously without the need to perform several sequential tests. Several studies have already validated the use of NGS and its superiority in term of sensitivity, speed and cost compared to traditional methods. [18,23,24] In our own experience, for tests including more than two to three different hotspots, NGS is cheaper, faster  and requires less DNA than would be needed for traditional methods. This is of particular importance for cytology samples and small-tissue biopsies for which several molecular alterations need to be screened, as for NSCLC samples e.g. [9,18]. Indeed, NGS requires only 10 ng for the full colon and lung cancer panel while traditional methods can require up to 10 ng of DNA for each mutation tested.
The precision (reproducibility and repeatability) analysis using reference standards with a known AF allowed us to show that all mutations with an AF >5% were consistently detected. However, mutations with an AF <3% were detected inconsistently. In addition, when clinical samples were analyzed 5 times in 3 different experiments, the Variant Caller inconsistently detected mutations with an AF <3% that are not detected by ddPCR, suggesting that these mutations correspond to sequencing artefacts, as is often observed with DNA extracted from FFPE samples [25][26][27]. The 4% threshold was thus selected for mutation reporting as a balance between maximizing the sensitivity and minimizing the false-positive results due to technical artifact. This threshold is consistent with other sensitivity and specificity data reported in the literature for this NGS platform [16,21]. Using this threshold, one case of NSCLC with a p. L858R EGFR mutation was missed. For this sample the Variant Caller detected the mutation at AF of 1.9%. However, given our criteria for considering a variant as authentic (AF >4% and variant coverage >30x), we could not validate this variant. It was recently proposed that known clinically relevant gene variants, such as EGFR mutations for NSCLC, should be reported irrespective of the AF [28]. In our current clinical practice, when we observe a known clinically relevant gene variant, but with an AF below the threshold, we report that the gene variant is suspected but not confirmed and that it would be interesting to test another sample from the patient if available.
Discrepancies between NGS and traditional methods were observed for 2 CRC cases with KRAS G12V mutations, both with an allelic frequency < 10%. This low AF can explain the discrepancies because the threshold of traditional method is varying from 3 to 20% of mutant DNA for the KRAS testing. For these 2 cases, the KRAS p.G12V mutations were validated by ddPCR.
One of the challenges using NGS is to interpret the detected mutations within the biological context. As already described [28], the variants can be grouped in three categories: i) those that may have a direct impact on patient care and are considered actionable; (ii) those that may have biological relevance but are not clearly actionable; and (iii) those that are of unknown significance. In the present study, NGS analysis detected mutations (other than EGFR mutations for NSCLC and than KRAS and NRAS for CRC) with potential clinical impact for 4 patients with NSCLC (one PIK3CA mutation and 3 STK11 mutations) and for 14 patients with CRC (10 PIK3CA mutations, 5 BRAF mutations-one patient harbouring PIK3CA and BRAF mutations). Indeed, preclinical data support the argument that NSCLC cell lines with PIK3CA or STK11 and KRAS mutation show increased sensitivity to PIK3 inhibitors or MAPK and mTOR signalling inhibition, respectively [29]- [30]. Moreover, a phase I dose-escalation clinical trial of a pan-class I PI3K inhibitor in patients with advanced solid tumors-primarily colorectal, breast, and lung-showed preliminary antitumor activity [31]. In the same way, apparent antitumour activity was observed for patients with BRAF mutated CRC treated with a selective mutant BRAF inhibitor [32].
In conclusion, the present study validated the clinical applicability of the Ion Ampliseq Colon and Lung cancer panel on the Ion Torrent Personal Genome Machine for screening lung and colorectal cancers. Overall, the AmpliSeq colon/lung cancer panel was specific and sensitive enough for mutation analysis of gene panels and can be incorporated into clinical daily practice.