Utility of a ready-to-use PCR system for neuroendocrine tumor diagnosis

Background Multigene-based PCR tests are time-consuming and limiting aspects of the protocol include increased risk of operator-based variation. In addition, such protocols are complex to transfer and reproduce between laboratories. Aims Evaluate the clinical utility of a pre-spotted PCR plate (PSP) for a novel multigene (n = 51) blood-based gene expression diagnostic assay for neuroendocrine tumors (NETs). Methods A pilot study (n = 44; 8 controls and 36 NETs) was undertaken to compare CQ, normalized gene expression and algorithm-based output (NETest score). Gene expression was then evaluated between matched blood:tumor tissue samples (n = 7). Thereafter, two prospective sets (diagnostic: n = 167; clinical validation: n = 48, respectively) were evaluated for diagnostic and clinical utility value. Two independent molecular diagnostics facilities were used to assess assay reproducibility and inter-laboratory metrics. Samples were collected (per CLIA protocol) processed to mRNA and cDNA and then either run per standard assay (liquid primers) or on PSPs. Separately, matching plasma samples were analyzed for chromogranin A (CgA). Statistics included non-parametric testing, Pearson-concordance, Predictive Modeling and AUROC analyses. Results In the pilot study (n = 44), CQ values were highly concordant (r: 0.82, p<0.0001) and normalized gene expression data significantly related (p<0.0001) (Pearson-pairwise correlation). NETest values were not different (49.7±33 standard vs. 48.5±31.5 PSP) and the overall concordance in output 96%. Predictive modelling confirmed this concordance (F1 score = 0.95). Gene expression levels were highly correlated between blood and tumor tissue (R: 0.71–0.83). In the diagnostic cohort (n = 30 controls, n = 87 non-NET controls, n = 50 NET), NETest was significantly lower (p<0.0001) in controls (11±6.5) and non-NET controls (13±18) than NETs (61±31). The AUROCs were 0.93–0.97 and the diagnostic accuracy was 90–97.5%. As a diagnostic, the PSP-NETest was significantly better than CgA (accuracy: 56%, p<0.0001). For clinical samples, the PSP generated robust and accurate (>96%) scores and was significantly better (p<0.0001) than CgA. The assay protocol was consistent (r: 0.97) and reproducible (co-efficient of variation: 1.3–4.2%) across the two facilities. Conclusion The PSP protocol for the NETest has been established and prospectively tested in clinical samples. It is highly reproducible, has similar metrics (CV, categorization by control or NET) to the standard PCR assay and generates clinically concordant (>96%) NETest results. Moreover, it functions significantly more accurately than CgA.


Introduction
Critical issues in biomarker measurement are the requirement that the assay is highly reproducible, can be standardized and can be conducted in different laboratories with equal facility. This is particularly important when complex novel strategies are under development and assessment. Real-time PCR (qPCR) is the gold standard for nucleic acid quantification. Its strengths include sensitivity, a wide dynamic quantitative range, and the ability to reliably differentiate small differences in gene expression. The clinical utility of this strategy has been established and tests based on qPCR of selected genes e.g., MammaPrint or the Oncotype Dx family of assays, have become the standard for determining prognosis and specific therapeutic choice in oncology [1][2][3][4][5][6]. We developed and clinically validated a qPCR-based gene expression assay (TaqMan primer/probe sets from ThermoFisher) for diagnosing and managing gastroenteropancreatic neuroendocrine tumors (NETs) based on a 51-marker gene signature [7][8][9][10]. The assay-NETest-is undertaken in a CLIA-approved molecular diagnostics facility (Wren Laboratories, CLIA 07D2081388) and has been evaluated and the protocol and test accepted by the New York Department of Health (PFI: 9138). The assay is significantly more accurate than other biomarkers like chromogranin A (CgA) [11]. The NETest assay is based on isolation of circulating mRNA from whole blood, cDNA synthesis and then real-time PCR against 51 targeted genes that capture the biology of the neuroendocrine tumor cell [9]. The methodologies are standard but time-consuming since individual assays are run in 384-well plates.
Ready-to-use PCR plates, known as pre-spotted plates (PSP), consist of plastic supports for qPCR (e.g. 96-or 384-well plates) in which primers and probes from chosen assays are predispensed and dried on plates. PSP were developed to facilitate multi-target analyses. Thus, a single reaction mixture per sample, containing all reagents except primers and probes, is prepared and loaded into the appropriate wells. This approach accelerates laboratory productivity and decreases the likelihood of error. An additional advantage is that it allows for a straightforward transfer of a qPCR-based assay to a collaborating laboratory facility that possesses molecular diagnostic skill-sets or the ability to automate the PCR-step. Successful examples range from the development of PSPs to screen for genetically modified organisms (GMOs) in food samples [12] to assays for multi-pathogen detection [13] to a 73 gene-based assay that evaluates the entire coding region of BRCA1 and BRCA2 [14]. These strategies were developed as time-and cost-effective methodologies that could produce reliable, reproducible results and are straight-forward to implement.
The exponential nature of PCR combined with small quantities of target or mispriming events, however, can result in substantial differences in the final result [15][16][17][18][19]. These considerations [20,21] underscore the importance of stringent quality control during the validation of qPCR-based tools [22]. Wren Laboratories in collaboration with ThermoFisher constructed a specific NETest-PSP based on the 51-marker genes, and positive/negative controls established and required for CLIA-approved clinical testing. We evaluated the utility of these preloaded plates to run the NETest assay in a pilot and validation series of samples. We focused on the concordance of gene expression and output (algorithmic results) to stringently determine the utility of the PSPs. We then tested the assay in two different molecular diagnostic laboratories in separate countries as a further test of robustness of the assay. We also compared the assay to CgA in matched samples to verify its diagnostic utility. Finally, we examined whether the PSP could adequately replace the TaqMan-based assays for CLIA clinical samples by evaluating output in prospective, clinically requisitioned samples.

Study design
Several interlinking approaches were used to investigate the utility of PSPs. These, per STARD2015 criteria [23], are included (Fig 1). All samples were collected and analyzed according to an IRB specifically approved protocol (Yale University School of Medicine: HIC0805003870, approved 6/15/2016). Informed written consent was obtained from all study subjects. Blood and tumor tissue samples were collected prospectively between July 2017 and April 2019 from patients and non-affected family members (controls) of the patients attending the NET outpatient center or from patients undergoing clinical NETesting at Wren Laboratories (WIRB: 20150174, approved: 2/9/2015).
Briefly, 44 retrospectively collected samples (8 controls and 36 NETs) were evaluated using liquid approach and PSPs. The C Q and normalized gene expression levels were compared to identify the utility of the plates to amplify target genes accurately. The output (NETest) was examined and compared to determine the reliability of the PSPs to generate an accurate score. Thereafter, a comparison was undertaken in matched blood and tumor tissue samples (n = 7) to verify that the PSP-NETest functioned as a liquid biopsy. One hundred and sixty-seven independently collected samples (30 controls, 87 non-NET controls and 50 NETs; samples deidentified before evaluation) were then tested on PSPs and the diagnostic utility of the test evaluated. Matched plasma samples were available for CgA analysis. Then, an inter-laboratory concordance evaluation was undertaken in two separate, accredited molecular diagnostic laboratories to investigate the robustness of the PSPs (n = 18 samples; deidentified). Laboratory metrics including inter-and intra-laboratory assessments were then undertaken. Finally, the PSP was prospectively examined in 48 consecutive clinical samples to establish a PSP protocol for a clinical laboratory. To further validate the clinical utility of the PSP protocol, we compared the NETest results with those of CgA undertaken at the same time-point.

Clinical samples
Blood samples for the NETest were collected in 10.8mg K 2 EDTA tubes (BD Vacutainer Venous Blood Collection Tubes, BD Diagnostics). Aliquots of whole blood were stored at -80˚C within 2 hrs of collection (samples immediately stored on ice/4˚C after sampling) per standard molecular diagnostics protocols [24]. Blood samples for CgA were collected at the same time point in PPT plasma preparations tubes. Matched tumor tissue samples were available in seven cases. Tissue (n = 7) were collected at the time of surgery [25]. Samples were snap frozen in liquid nitrogen. Deidentified samples were sent to Wren Laboratories for RNA isolation and NETest PCR.
Sample Set I. Eight healthy controls were measured by standard and spotted plate PCR approaches respectively. Similarly, n = 36 NETs (lung: n = 4; pancreatic: n = 12; small bowel: n = 18; appendiceal n = 2) were assessed using both techniques. Matched tumor tissue was available from 7 GEP-NETs for analysis. This included 3 pancreatic NET and 4 small bowel NET.

Sample preparation and PCR
The studies were undertaken in a USA clinically certified laboratory (Wren Laboratories CL-0704, CLIA 07D2081388, New York Department of Health: PFI 9138, NETest: 55835). Sample Set III was also independently evaluated in the SCMD Facility, London, UK). The details of the PCR methodology, mathematical analysis and validation have been published in detail [26][27][28]. A 3-step protocol (RNA isolation with cDNA production and qPCR) was undertaken using a manual technique in the authors' laboratory. Transcripts (mRNA) were isolated from 1ml EDTA-collected blood samples using the QIAamp RNA Blood Mini Kit (Qiagen). The RNA quality (A 260:280 ratio) and quantity are measured using NanoDrop. The standard Qiagen isolation protocol (heme/gDNA contamination not detected) with no modifications was used. cDNA was produced from 50ul of eluted RNA using a High Capacity Reverse Transcription kit (Applied Biosystems: cDNA production 2000-2500ng/ul) and stored at -80˚C.
Standard assay. Real-time PCR was performed (384-well plate) with the cDNA (200ng) and 16ul of reagents/well (Universal Master Mix II with UNG, Applied Biosystems, triplicate wells) (50˚C 2 min, 95˚C 10 min, then 95˚C 15s, 60˚C, 60s for 40 cycles). The majority of the 51 primers are exon spanning (82%, the remainder (18%) spanned one exon); all were <160bps (Table 1). For clinical testing a 7900HT qPCR machine was used. For all other testing, QuantStudio 7 Flex qPCR machines were used. PSP assay. Real-time PCR was performed with the cDNA (200ng) and 8ul of reagents/ well (Universal Master Mix II with UNG, Applied Biosystems).
CgA ELISA. This was measured using NEOLISA Chromogranin A kit (Euro Diagnostica AB, Malmö, Sweden) at Wren Laboratories. The upper limit of normal was 108 ng/ml [28]. The laboratory is CLIA (07D2081388) and NYDOH certified (55834) for undertaking CgA clinical testing.
Statistics. In the pilot study (Sample Set I), intergroup analyses were undertaken using 2-tailed non-parametric tests (Mann-Whitney U test), Pearson correlations were used (for matched C Q or gene expression samples: standard/PSP) and Predictive Modeling (PM) also undertaken. For PM, a historical NETest dataset [7] consisting of n = 158 Healthy Controls and n = 134 NETs was used to generate training and testing sets. Extreme Gradient Boosting (Xgboost), a supervised learning algorithm, was used to generate training and testing sets. Xgboost model was trained on a randomly generated stratified subset of samples consisting of 75% of all data, whilst the remaining 25% of data was used to assess model performance. Model hyper-parameter optimization was undertaken using Bayesian Optimization [30]. Briefly, 5-fold cross validation was performed to assess hyper-parameter effects and optimization was carried out on minimum child node weight, column subsamples per tree, maximum tree depth, subsample size, gamma, and alpha parameters. Learning rate was set to a constant 0.1 and training was undertaken over 3000 epochs with an early stopping at 50 epochs in cases where performance would not improve. Mean Absolute Error was used to assess model fitness. For tumor:blood pairs, normalized data were log-transformed and Pearson regression analysis undertaken as described [25]. In the Diagnostic group (Sample Set II), Mann-Whitney U-test and Area under the Receiver Operator Curve (AUROC) analysis was used to determine the diagnostic accuracy of the NETest [31][32][33]. Metrics calculated included sensitivity and specificity. To compare the utility of the PSP with CgA, McNemar's test was undertaken in matched samples (NETs and non-NET controls) while diagnostic accuracies were directly compared (comparative ROC analysis) in the same groups.
In the inter-laboratory (Sample Set III) and the clinical validation (Sample Set IV) series, Mann-Whitney U-test and Pearson correlations were utilized. Inter-and intra-assay co-efficient of variation was undertaken per standard laboratory techniques [8]. Utility of the PSP to diagnose the NETest was directly compared to CgA using Fisher's exact test.

RNA yield and quality
We initially examined the yield and efficiency of RNA extraction. The RNA yield (n = 259 samples) was 12

Concordance between standard plates and PSP for C Q and normalized gene expression
We next compared C Q values and normalized gene expression levels between the standard (liquid) and PSP approaches. We evaluated paired samples and calculated the Pearson values for the C Q for each gene (Fig 2A and 2B) and the normalized gene expression (Fig 2C and 2D). These were highly and significant (p<0.001) concordant for the C Q (0.76±0.18 [0.82:0.69-0.86]) (Fig 2A and 2B). Likewise, for normalized gene expression data, the Pearson r was highly concordant (p = 0.0048 -p<0.0001) (0.65±0.18 [0.65:0.54-0.76]) (Fig 2C and 2D).

NETest (algorithm output) subgroup analysis
Thereafter, we used two methods to evaluate whether the NETest scores from the PSP differentiated between controls and NETs. In the first approach, we audited the forty-four samples from Sample Set I and, using the set cut-off threshold of 20 (on a scale of 0-100), evaluated how many samples fell into each category (normal: �20 or "NET" >20). The NETest scores for controls were completely concordant between the liquid approach and PSP-each identified 8/8 (100%) controls with scores less than 20 (Fig 3). All 36 NETs (100%) demonstrated NETest scores >20 using the standard approach. Thirty-four (94%) also had high NETest scores using the PSPs. The overall level of concordance between the standard approach and the PSP, therefore, was 42 of 44 (95.5%).
In the second method, we used Predictive Modeling (xgboost) to directly compare the NET vs. Control classification probabilities (liquid versus PSP) from Sample Set I against the initial set used to develop the original algorithm in 2013. This historical NETest dataset [7] consisted of n = 158 Healthy Controls and n = 134 NETs. Both standard PCR and spotted plate techniques differentiate NETs from Healthy Controls with comparable accuracies (average F1 Score of 0.98 and 0.95 for standard and spotted techniques respectively, Table 2).

Concordance between gene expression in matched blood and tumor tissue
Finally, we evaluated the concordance between gene expression in matched tumor tissue and blood sample pairs (n = 7, including PNET: n = 3; small bowel NET: n = 4). The Pearson correlation (R) for the normalized gene expression in each of the tumor-blood paired groups was 0.71, and the Spearman correlation was 0.83 ( Fig 3C). This was highly significant (p<0.0001).    (Fig 4A). An area under the curve analysis identified an AUC of 0.97 for controls and 0.93 for non-NET controls. The Z-statistics were 27.1 and 19.9, while the Youden J indices  were 0.93 and 0.82, respectively. The PSP exhibited a sensitivity of 96% and a specificity of 100% using a cut-off of 20 for controls. The sensitivity for non-NET controls was 96% with a specificity of 86% using the cut-off of 20. The overall concordance between distinguishing sample types was 153/167 (91.6%). As a second evaluation, we compared the PSP-NETest results with the CgA results in matched samples. CgA levels were positive in three (10%) of controls, in 25 (29%) of non-NET controls and in 22 (44%) of NETs (Fig 4C). The AUROC for differentiating controls from NETs was 0.71; for non-NET controls, it was 0.56. A direct comparison between CgA and PSP-NETest identified the latter was significantly more accurate (123/137, 90%) than CgA alone (74/137, 54%) for differentiating NETs from non-NET controls. The McNemar's test was highly significant (Chi 2 = 20.0, p = <0.0001). A comparison of AUROCs confirmed that the PSP-NETest functioned significantly better than CgA for differentiating NETs from non-NET controls (difference in AUCs: 0.38, z-statistic: 6.44, p<0.0001; Fig 4D).

Inter-laboratory concordance
To assess the robustness of the PSP-based NETest, two clinical laboratories (one from the USA and one from the UK) each tested 18 "real-life" matched blood samples (Sample Set III). Firstly, we examined the RNA quality metrics between the two laboratories. The RNA quantity was 5.5±2.6ng/ml (Laboratory 1 -Wren)  The Wilcoxon matched pairs signed rank test identified a p = 0.328. The Pearson correlation r was 0.967 (95%CI: 0.935-0.99), p<0.0001 (Fig 5B).

Inter-and intra-laboratory reproducibility
For inter-assay reproducibility, multi-day (5-10 different days) reproducibility was examined. This was undertaken in 5 different clinical samples (Sample Set III). The metrics (co-efficient of variation) ranged from 0% to 10.4% (Fig 6A). The averaged CV was 4.24%. This value is similar to that defined for the liquid assay (5.9%) [8]. Between-samples/plate (intra-assay) variation was then evaluated in 30 clinical samples. The averaged co-efficient of variation was 1.26% (Fig 6B). This value is almost identical to that defined for the liquid assay (1.2%) [8].

Clinical Testing Evaluation
Finally, to assess the robustness of the PSP-based NETest, we directly compared NETest scores from liquid PCR and PSPs. Bloods were from 48 consecutive NET patients (Sample Set IV) ordered and undertaken under CLIA laboratory conditions and reported for clinical decisionmaking. NETest scores were 38.6±21.6 [26.7:26.7-40] (standard method, CLIA-reported) and 40.8±22.1 [33.3:26.7-40] (PSPs) (Fig 7A). The Wilcoxon matched pairs signed rank test identified a p = 0.675. The Pearson correlation r was 0.94 (95%CI: 0.899-0.968), p<0.0001 (Fig 7B). An examination of the variability across all the 51 marker genes identified this was significantly lower in the PSP-evaluated samples than for the standard liquid approach (median: 7.9 vs. 12.8, p = 0.0059).
An evaluation of outputs (NETest scores) identified 96% concordance between the standard approach and PSPs ( Table 3). One of the samples categorized into the "low" score category (40) had an "intermediate" score (53) on the PSP, while a second sample with an intermediate score of 73 was called a high-score of 80 -on the PSP. As a second evaluation, we compared the PSP-NETest results with CgA results (Fig 7C) in matched patient samples. More patients were NETest-positive (98%, 47/48) compared to CgApositive (27%, 13/48). This was highly significant (Fisher's exact test: p<0.0001).

Discussion
The purpose of this manuscript was to standardize and establish the metrics of the PSP protocol with the aim of replacing the current liquid-based, CLIA-approved qPCR protocol. Our results confirm that a PSP platform can provide an acceptable replacement for the liquidbased NETest qPCR assay. The results demonstrated that the pre-spotted plates generated equivalent gene expression levels and outputs (scores) and were highly correlated with tumor tissue levels. Observations from the pilot study were independently validated and the diagnostic accuracy for the plates was confirmed to be >90%. The assay based on the PSPs was robust, standardized and easily transferrable to an independent molecular diagnostic facility. Furthermore, the PSPs generated clinically valid data concordant with liquid PCR under CLIA conditions. Finally, the PSPs, like the liquid-based NETest, was significantly more accurate than CgA as a diagnostic.
In our pilot study (n = 44), we examined the C Q values between the standard (liquid) and PSP approaches and identified that these were highly concordant (r: 0.82) while the  normalized gene expression was effectively reproduced on the plates (p<0.0001). An examination of the final output-gene expression conversion to a score-demonstrated that levels were highly correlated (r: 0.87) and not significantly different between the two approaches. An audit of these samples identified that the NETest scores for controls were completely concordant while almost all (34/36 or 94%) NETs studied had NETest scores >20 i.e., were called "tumor". The overall level of concordance between scores was 95.5%. To independently confirm this, we compared the NETest scores against the original historical set used to develop the algorithm using Prediction Modeling. The average F1 Score for differentiating NETs from controls was 0.98 (standard) and 0.95 (PSP), confirming that the PSP technique was as accurate as the standard PCR approach. To validate that the PSP-NETest was an effective liquid biopsy, we directly compared matched tumor tissue and blood samples. The concordance was highly significant (R: 0.71-0.82, p<0.0001) and similar to what has been previously reported for the liquid-based assay [25,34]. These data confirm that the PSP detects circulating tumor transcripts and functions as a bona fide liquid biopsy. We next evaluated the utility of the PSP to differentiate between controls and NETs that were prospectively and independently collected. The NETest scores in the 30 known controls were <20 (mean: 11). In the 87 non-NET controls-including subjects with non-NET diseases that have been associated with clinical neuroendocrine features or a neuroendocrine phenotype (including, GERD, IPMN, pancreatic cysts, lung, pancreatic and colon adenocarcinoma), 75 were <20 (mean: 13). Two IPMN, one cystadenoma, 1 pancreatic adenocarcinoma, 2 colon cancers 2 lung adenocarcinomas and two squamous cell carcinoma were NETest-positive. Five of these NETest-positive samples also had elevated CgA levels. The positive NETest findings in IPMN and colon adenocarcinoma are consistent with observations from an earlier publication that identified minor overlap [35]. Similarly, positive NETest scores in lung neoplasia have recently been reported based upon the existence of neuroendocrine elements in lung cancer [36]. A histopathologically verified colon NEC and a G3 gastric NEC were both NETest-positive (score: 87 and 80, respectively) as would be predicted in tumors that have a neuroendocrine genotype and phenotype. The NETest genes have previously been independently confirmed as expressed in neuroendocrine carcinoma tumor tissues in a large NIH study of >10,000 tumors [37]. These authors (Chen et al.) also reported that 30-40% of histologically defined adenocarcinomas in the lung and gut expressed NET genes. This would be consistent with our observation that the two NEC patients would exhibit positive blood levels. Similarly, it is not unexpected that some lung and gut neoplasia will be NETest-positive. In the 50 NETs, the score was significantly (p<0.001) elevated (mean of 61) as compared to the non-NET conditions (mean of 13).
Forty-eight (96%) of the 50 were NETest-positive. Two patients, a rectal NET-5 years postsurgery and a lung NET [typical carcinoid] 3 years post-surgery-both of whom had no evidence of disease, were therefore correctly identified. The AUROC analysis confirmed the test results from PSPs could be used as a diagnostic-the AUC was 0.97 versus controls and 0.93 versus non-NETs. Using the standard cut-off of 20 resulted in a diagnostic sensitivity of 96% and a specificity of 86-100% for the PSPs. The overall concordance between distinguishing sample types was 92%. In contrast, CgA measured in matched samples collected at the same time points was elevated in only 44% of NETs. The PSP-NETest, like the liquid-based PCR assay, was therefore confirmed to be significantly more accurate than a single analyte assay.
We next evaluated the robustness of the PSP-based NETest by comparing the process in two molecular diagnostics facilities, one in Branford, USA, the second in London, UK. Both facilities were provided with aliquoted blood samples (identical samples: n = 18). Each independently processed (from blood) and undertook cDNA synthesis and qPCR. NETest scores were not significantly different (mean: 42 vs. 43) and highly concordant (r: 0.97). This demonstrates that the protocol for diagnosing a NET using blood-gene expression is robust and that the PSP protocol can be adequately and appropriately transferred to other molecular technology familiar laboratories. The robustness of PSP approaches has previously been confirmed for a GMO PCR-screening assay. Nineteen different laboratories (throughout Europe) evaluated a 10-gene PSP to screen for GMOs in food samples [12]. The concordance ranged between 92-99%. Our inter-laboratory evaluation (97%) generated analogously accurate results.
We have previously evaluated the inter-and intra-assay metrics for the standard assay [8] demonstrating the inter-assay CV to be 5.9% and the intra-assay CV to be 1.2%. The assay on PSPs was as highly reproducible; the inter-assay CV was 4.24% and the intra-assay variation was 1.26%. In the UK laboratory, inter-and intra-assays were 2.2 and 1.2% respectively. These values demonstrate that the plates exhibit metrics that are at least as good as the standard assay [8]. Furthermore, an examination of the variability across all the 51 marker genes (Sample Set IV) identified that inconsistency was significantly lower in the PSP-evaluated samples than for the standard approach (median: 7.9 vs. 12.8, p<0.006). This further supports the consideration that pre-spotting PCR primers reduces the technical variability that may occur in liquid-based assays. This is especially likely when large numbers of genes (n = 51) are being evaluated.
Finally, to assess the robustness of the PSP-based NETest, we directly compared NETest scores from 48 consecutive NET patients and compared this to CgA. These were ordered and undertaken under CLIA laboratory conditions and reported for clinical decision-making to the same samples run on PSPs. Firstly, the PSP-NETest was identified to be significantly more effective than CgA as a diagnostic (p<0.0001). Secondly, NETest scores were concordant (mean: 39, CLIA-reported versus mean: 41, PSPs) and highly correlated (0.94), p<0.0001. Of note, the one clinical sample identified to have a normal NETest-a rectal NET two years after surgery and known to have no image or clinical evidence of disease-was appropriately identified by the PSP approach. We also examined the concordance based on categorizing the outputs into normal, low, intermediate and high categories of score-per CLIA and NYDOH requirements. The PSP results were highly concordant-46/48 (96%)-with the standard approach. Two samples were misclassified. One sample exhibited a low score (CLIA/standard plate) of 40. On the PSP, the score was 53 which is a category change-from low to intermediate. The accuracy for "low" scores therefore is 97%. In this instance, the category change would be expected to result in a more careful follow-up of the patient [10]. The second sample had an intermediate score of 73 (CLIA/standard plate) whereas on the PSP, the score was 80 -a category change from intermediate to high. High scores are associated with a poorer prognosis [10]; in this instance, the physician would be alerted to the high score and the potential for an early relapse. PSP in two instances thus upgraded the risk category suggesting that this strategy might be effective in increasing the clinical vigilance required. A large study with far greater clinical patients would be required to assess this increase in efficacy.
In conclusion, the new spotted plate protocol for the NETest has been verified and prospectively tested in clinical samples. The spotted plate protocol is highly reproducible, has similar metrics (co-efficient of variation, concordance in output and overall categorization by control or NET) to the liquid assays, generates concordant NETest results and, like the liquid-based assay, functions significantly more accurately than CgA as a NET diagnostic. This approach is technically simpler than the standard liquid-based approach, uses less reagents and is more reproducible than standard protocols.
There is an increasing interest in different forms of liquid biopsies that can be utilized to guide patient care. Since such approaches are providing significant benefit and becoming increasingly integrated into clinical practice, we anticipate that the NETest will become a standard of care diagnostic for NET clinical management [38]. The NETest has demonstrable utility as a disease diagnostic. In addition, it can assess the effectiveness of surgery by the identification of residual or recurrent disease [27,39,40]. The NETest can also distinguish stable from progressive disease and monitor the effectiveness of medical treatment [10]. Given that our approach has been independently validated [34,37], we anticipate that usage will become widespread. Clinical application and usage of this molecular assay will be facilitated by the PSP approach we describe. We anticipate that a PSP-NETest will provide both a significant cost savings for gene expression assays and allow for a broad introduction of the NETest assay into clinical laboratory practice.