https://orcid.org/0000-0001-5782-9950
Figures
Abstract
Combining diagnostic specimens into pools has been considered as a strategy to augment throughput, decrease turnaround time, and leverage resources. This study utilized a multi-parametric approach to assess optimum pool size, impact of automation, and effect of nucleic acid amplification chemistries on the detection of SARS-CoV-2 RNA in pooled samples for surveillance testing on the Hologic Panther Fusion® System. Dorfman pooled testing was conducted with previously tested SARS-CoV-2 nasopharyngeal samples using Hologic’s Aptima® and Panther Fusion® SARS-CoV-2 Emergency Use Authorization assays. A manual workflow was used to generate pool sizes of 5:1 (five samples: one positive, four negative) and 10:1. An automated workflow was used to generate pool sizes of 3:1, 4:1, 5:1, 8:1 and 10:1. The impact of pool size, pooling method, and assay chemistry on sensitivity, specificity, and lower limit of detection (LLOD) was evaluated. Both the Hologic Aptima® and Panther Fusion® SARS-CoV-2 assays demonstrated >85% positive percent agreement between neat testing and pool sizes ≤5:1, satisfying FDA recommendation. Discordant results between neat and pooled testing were more frequent for positive samples with CT>35. Fusion® CT (cycle threshold) values for pooled samples increased as expected for pool sizes of 5:1 (CT increase of 1.92–2.41) and 10:1 (CT increase of 3.03–3.29). The Fusion® assay demonstrated lower LLOD than the Aptima® assay for pooled testing (956 vs 1503 cp/mL, pool size of 5:1). Lowering the cut-off threshold of the Aptima® assay from 560 kRLU (manufacturer’s setting) to 350 kRLU improved the assay sensitivity to that of the Fusion® assay for pooled testing. Both Hologic’s SARS-CoV-2 assays met the FDA recommended guidelines for percent positive agreement (>85%) for pool sizes ≤5:1. Automated pooling increased test throughput and enabled automated sample tracking while requiring less labor. The Fusion® SARS-CoV-2 assay, which demonstrated a lower LLOD, may be more appropriate for surveillance testing.
Citation: Park R, Chandrasekaran P, Hernandez H, Lakhal-Naouar I, Peachman KK, Hack HR, et al. (2022) Validation of SARS-CoV-2 pooled testing for surveillance using the Panther Fusion® system: Impact of pool size, automation, and assay chemistry. PLoS ONE 17(11): e0276729. https://doi.org/10.1371/journal.pone.0276729
Editor: Etsuro Ito, Waseda University: Waseda Daigaku, JAPAN
Received: July 29, 2022; Accepted: October 12, 2022; Published: November 7, 2022
This is an open access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication.
Data Availability: The data are available in the Dryad Repository: https://doi.org/10.5061/dryad.n5tb2rc04.
Funding: This work is supported in part by the US Army Medical Research and Development Command under Contract No. W81-XWH-18-C-0337 and by the US Military HIV Research Program, Walter Reed Army Institute of Research under a cooperative agreement under Contract No. W81XWH-18-2-0040 between the Henry M. Jackson Foundation for the Advancement of Military Medicine, Inc., and the U.S. Department of Defense (DoD). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. No author received a salary from any of the funders.
Competing interests: The authors have declared that no competing interests exist.
Introduction
The dire need for early detection and surveillance of SARS-CoV-2 infections led laboratories and global diagnostic manufacturers to rapidly develop approaches for virus detection [1, 2]. The Food and Drug Administration (FDA) has subsequently permitted the use of over 269 molecular assays under its Emergency Use Authorization (EUA) mechanism [3]. In order to contain the SARS-CoV-2 pandemic, surveillance testing has been implemented so that asymptomatic individuals can be identified, treated, and isolated to limit further spread [4, 5]. Asymptomatic spread is uniquely relevant to SARS-CoV-2, as most other disease infections are symptomatic with the onset of viremia [6].
One strategy to expand surveillance testing is to implement pooled sample testing, which involves combining multiple samples into a single test to reduce the turnaround time, cost, and laboratory resources required [7–9]. The performance of pooled testing depends on multiple parameters including disease prevalence, pool size, pooling method, and assay chemistry [10]. Pool sizes up to 64 [11–13] have been recommended for SARS-CoV-2 surveillance testing in low disease prevalence conditions. Automated pooled sample generation can further increase surveillance testing throughput and can reduce pipetting errors and cross contamination, but it requires investment in facility preparation, liquid handling capability, and software development and verification.
In addition to pool size and pooling method, assay chemistry can influence pooled testing performance. Currently, real-time reverse transcription-polymerase chain reaction (RT-PCR) assays are the most common type of molecular test for SARS-CoV-2 diagnosis. However, transcription mediated amplification (TMA) is an alternative nucleic acid amplification chemistry that offers some advantages over RT-PCR. TMA utilizes RNA polymerase to generate many RNA transcripts from a DNA template at a fixed temperature, unlike RT-PCR which uses a temperature sensitive DNA polymerase to generate a single copy of a DNA template per thermal cycle.
This study was designed to evaluate automated pooled testing for surveillance of SARS-CoV-2 infections in the Military Health System. We compared a range of pool sizes using manual and automated methods for pooling samples followed by both RT-PCR and TMA assay testing. Specifically, we evaluated pooled sample testing with both chemistries using the Hologic, Inc. (Marlborough, MA, USA) Panther Fusion® SARS-CoV-2 assay (real-time RT-PCR) and the Hologic Aptima® SARS-CoV-2 assay (TMA). Both molecular assays are approved for use under EUA by the FDA and were run on the same Panther Fusion® system.
Materials and methods
Clinical discard samples
For the manual pooling study, 1652 post residual clinical test samples were received under a minimal risk Human Subjects Research study approved by the Walter Reed Army Institute of Research (WRAIR) Institutional Review Board (WRAIR #2810). Nasopharyngeal (NP) swabs were collected in Viral Transport Media (VTM). Due to supply chain disruptions multiple VTM sources were used but all were cleared by the FDA. Diagnostic testing was performed at the Naval Infectious Diseases Diagnostics Laboratory (NIDDL) within the Naval Medical Research Center (NMRC), using CDC 2019-Novel Coronavirus Real-Time RT-PCR Diagnostic Panel or ThermoFisher TaqPath™ COVID-19 Combo Kit assays. Post residual test samples were stored at -80°C, then transferred to the Department of Research, Assessment and Development, Diagnostics and Countermeasures Branch, Center for Infectious Diseases, Walter Reed Army Institute of Research. In total, 106 SARS-CoV-2 positive and 308 negative post residual samples were used for testing. To evaluate assay specificity, 32 negative samples (28 from NIDDL and 4 purchased from BioChemed (Winchester, VA)) were also subjected to pooled testing. For the automated pooling study, 2028 de-identified post residual test samples were acquired from Walter Reed National Military Medical Center (WRNMMC). The EUA Roche cobas® SARS-CoV-2 test was used as the reference clinical diagnostic assay. A total of 107 positive samples and 450 negative samples were used for generating pools; details of the samples are listed in S1 Table. Both automated and manual pooled testing schemas used samples with CT ranging from 15 to >35, with ~15% low viral load samples (CT>35). The samples used to support automated pooling came from a cohort with a higher proportion (56%) of moderate viral load samples (25<CT<35) than the cohort that supported the manual pooling study (35% moderate viral load).
Manual sample pooling
Previously identified positive SARS-CoV-2 post residual clinical test samples were tested neat and in pool sizes of 5:1 and 10:1. For neat testing, 500 μL of each sample was pipetted directly into a Panther Fusion® Specimen Lysis Tube (SLT) containing 0.71 mL of Specimen Transport Medium (STM). For pooled sample testing, a two-step pooling process was performed in which the negative samples were first pooled into a sterile, RNase/DNase-free tube, then pipetted into an SLT. Pools were completed by pipetting a positive sample into the SLT. All samples were pipetted in equal volume (100 μL for pools of 5:1, 50 μL for pools of 10:1) to create a final pool volume of 500 μL. A one-step pooling process was performed to determine if false positive results could be reduced by directly pipetting negative samples into SLTs. The schemas for manual pooling are presented in S1 Fig.
Automated sample pooling
Post residual clinical test samples were tested neat or in pool sizes of 3:1, 4:1, 5:1, 8:1 and 10:1. Neat and pooled samples were prepared using a Tecan (Zürich, Switzerland) Freedom EVO 150 robotic liquid handler equipped with an Air Liquid Handler and PosID3 barcode reader. The EVO 150 was housed inside a Labconco (Kansas City, MO) Logic Vue Class II enclosure. The software (scripts) required to automate pipetting with the Tecan EVO 150 were defined by the authors (DCB), written by Tecan’s Clinical Applications Specialists, and verified by the authors (DCB) and Tecan. The operator enters the pooling parameters (number of samples per pool, number of samples to be prepared, final pool volume), and the script performs the pipetting and sample tracking. Barcoded labels were affixed to the source sample tubes and pooled sample tubes to enable automated sample tracking.
Hologic Panther® SARS-CoV-2 molecular assays
Two high-throughput, automated SARS-CoV-2 molecular assays were used for this pooling study. The Hologic Panther Fusion® SARS-CoV-2 assay, which utilizes real-time RT-PCR and gives a semi-quantitative result (positive/negative and CT), and the Hologic Aptima® SARS-CoV-2 assay, which utilizes TMA [14] and yields a qualitative (positive/negative) result. Results for the Aptima® SARS-CoV-2 assay are expressed as the final amount of amplified target as kilo Relative Light Units (kRLU). The assay results are determined by a cut-off based on the total RLU (>560 kRLU for positive result) and the kinetic curve type. Both assays were performed on the same Panther Fusion® instrument, with neat and pooled samples tested simultaneously. Characteristics for the two Hologic assays are summarized in S2 Table.
LLOD panel
To evaluate the effect of pooling on the LLOD of both Hologic assays, SARS-CoV-2 samples with varying concentrations of SARS-CoV-2 RNA were generated using heat inactivated SARS-CoV-2 RNA (HK-87, American Type Culture Collection (ATCC), Manassas, VA). Twelve concentrations of RNA from 24,000 copies/mL to 16 copies/mL were prepared in Universal Transport Medium (UTM, brand of VTM trademarked by Copan). Six pool sizes were created using SARS-CoV-2 RNA negative post residual clinical test discards. S3 Table shows the 12 starting concentrations in the “Neat” row. STM/UTM was used to contrive dilution panel concentrations. The 12 starting concentrations (“Neat” row) were contrived by adding a known quantity of the heat inactivated SARS-CoV-2 virus in STM/UTM media. Negative post residual clinical samples were used to generate pools of 3:1, 4:1, 5:1. 8:1 and 10:1 from these 12 starting concentrations.
False positive testing
To further investigate FPs and prevent bias, negative sample pooled testing was performed using both Aptima® and Fusion® assays. Thirty two (32) samples that had previously tested negative (either at NIDDL or WRNMMC) and also tested negative in our hands with both the Aptima® and Fusion® assays were manually pooled (5:1 and 10:1) using the two-step pooling process. It was hypothesized that pooling directly into the lysis buffer might prevent potential cross reactivity or interference, so we also tested 29 samples (25 negative, 4 positive) with manual pooling (5:1 and 10:1) using both the one-step and two-step pooling processes. Once the automated pooling platform was verified, we performed automated pooling with 40 previously tested negative samples for all pool sizes using the two-step pooling process.
Statistical analysis
Samples were deemed positive or negative based on the consensus of the original clinical result, the Aptima® SARS-CoV-2 assay result, and the Fusion® SARS-CoV-2 assay result. Confidence intervals (CI) for the likelihood ratios, sensitivity, and specificity calculations, and predictive values are presented as Log10, exact Clopper-Pearson and logit CIs, respectively. The reliability for qualitative detection was assessed through Cohen’s kappa coefficient (κ), where values of κ define the following categories: slight (0.0 to 0.2), fair (0.21–0.4), moderate (0.41 to 0.60), substantial (0.61 to 0.8), and almost perfect (0.81–1.0). κ calculations were performed using Python. Passing-Bablok regression analysis was performed by programming in R, version 3.5.1, by using the method comparison regression (mcr) package. The bar graphs and Bland-Altman plots were generated using GraphPad Prism, version 8.4.3. A Freidman test was performed to determine the significance of observed differences in the bar graphs. Statistics on the bar graphs and Bland-Altman plots were performed using GraphPad Prism. CIs were calculated using MedCalc, version 19.4.1. Statistics for Tables 1–4 were calculated using Microsoft Excel. Statistics for Tables 5 and 6 were calculated using Microsoft Excel and MedCalc. All comparisons were two-sided with type 1 error set to 0.05; hence, p-value<0.05 was considered significant.
Results
Aptima® and Fusion® SARS-CoV-2 RNA detection vs other SARS-CoV-2 EUA assays for neat sample testing
The neat sample results for both Hologic assays were compared with the previous results from NIDDL and WRNMMC for neat sample testing. The Fusion® CT values were comparable to neat CT values from the CDC 2019-nCoV (manual pooling, Fig 1A), TaqPath™ (manual pooling, Fig 1B) and cobas® (automated pooling, Fig 1C) assays. The Aptima® assay demonstrated 100% percent positive agreement (PPA) with CDC and TaqPath™ results, and 82% PPA with cobas® results (Fig 1D–1F). There was 93% PPA between the Aptima® and Fusion® assays. The Aptima® kRLU values have a nonlinear relationship with the quantitative Fusion® results (Fig 1G).
CT results for Fusion® (ORF1ab target) vs (A) CDC 2019-nCoV (N1 and N2 targets), (B) TaqPath™ (ORF1ab) and (C) cobas® (ORF1ab). Linear regression equation shown as y = R2 = Coefficient of regression. kRLU results for Aptima® (ORF1ab) vs (D) CDC 2019-nCoV, (E) TaqPath™ and (F) cobas®. (G) Aptima® vs Fusion® SARS-CoV-2 results with correlation coefficient (r) calculated for high (CT <25), moderate (25<CT<35) and low (CT >35) viral load ranges. Statistical significance (p) calculated by two-sided t-test. (D-G) Aptima® cut-off threshold (560 kRLU) shown in green.
Pooled testing PPA.
The Aptima® assay demonstrated >85% within test PPA between neat and pooled sample test results for all pool sizes (Table 1, top row both Manual and Automated Pooling). However, when comparing Aptima® automated pool sample results to results from the CDC, TaqPath™, and cobas® assays, PPA values decreased by 9.9–10.4% (Table 1, right, middle row). Reducing the cut-off for Aptima® from 560 kRLU to 350 kRLU led to improvement of overall PPA (>85%) for most pool sizes (Table 1, bottom row). Fig 2 shows the distribution of the Aptima® SARS-CoV-2 pooled sample test results.
Samples used in manual (left panel) and automated (right panel) pool generation. Test results are represented as circles. Numbers within bars represent median values. Dashed horizontal line represents cut-off value (560 kRLU). Two-sided t-test, * p<0.03, *** p<0.0001.
The PPA between pooled and neat testing for Fusion® was >85% for pool sizes ≤5:1, regardless of pooling method (Table 2, top row). For pool sizes >5:1, the PPA was <85% except for the manual pooling, 10:1 pool size scenario. The PPA results were similar when comparing the Fusion® pooled test results to neat test results with the CDC, TaqPath™ and cobas® assays (Table 2, bottom row). Fig 3 shows the distribution of the Fusion® SARS-CoV-2 neat and pooled sample test results. The median CT value increases as expected, and the distribution becomes more compressed for higher pool sizes.
Samples used in manual pool generation (left panel) and automated pool generation (right panel). Test results are represented as circles. Numbers within bars represent median values. Two-sided t-test, ***p<0.0001.
Samples were categorized into three groups based on their Fusion® CT values: (1) CT <25, high viral load; (2) 25 < CT < 35, moderate viral load and (3) CT > 35, low viral load (Fusion® LLOD CT = 35.6). Both assays performed well, regardless of pool size or pooling method. High and moderate viral load samples resulted in 94–100% PPA (Tables 3 and 4). Pooled test performance for samples with low viral loads was variable for both assays (Tables 3 and 4).
Test performance parameters for the Aptima® and Fusion® SARS-CoV-2 assays are presented with 95% confidence intervals in Tables 5 and 6. Both assays demonstrated high detection sensitivity, but the Aptima® assay sensitivity was lower in the automated pooling study. Specificity of the Fusion® SARS-CoV-2 assay was 4.3–10.1% lower than that of the Aptima® assay except for the 4:1 pool size which yielded identical specificity by both assays. Cohen’s kappa coefficient (κ) values showed almost perfect agreement for the Aptima® assay for pool sizes of 3:1, 4:1, 8:1 and 10:1 (Automated), with substantial agreement for all other pool sizes. The Fusion® assay demonstrated near perfect agreement for only the 5:1 (manual) pool size.
CT shift and linearity.
A Bland-Altman plot showed good agreement for Fusion® CT results between the neat and pooled samples for both testing schemas (Fig 4A and 4C). For the 5:1 pool size, the CT shift due to pooling is 1.91 (manual) and 2.41 (automated), while the 10:1 pool size had a CT shift of 3.03 (manual) and 3.29 (automated).
(A)(C) Bland-Altman plot and (B)(D) Passing-Bablok regression plots are presented for 5:1 or 10:1 pooled testing versus neat with the Panther Fusion® SARS-CoV-2 assay. For the Bland-Altman plots, the area between the dotted lines (green or orange) indicates the 95% limits of agreement. For the Passing-Bablok regression, the confidence interval is shaded (green or orange) and the blue line indicates the line of identity. The slope and intercept of the regression line are reported in the left top of each panel.
Passing-Bablok regression fit indicates good linearity (slope confidence interval), with good agreement between the neat and pooled results for both 5:1 and 10:1 pool sizes (p<0.0001, Fig 4B and 4D).
Source of RNA impacts LLOD
In the heat inactivated virus + STM/UTM case, Aptima® demonstrated a lower LLOD (55 copies/mL) than Fusion® (108 copies/mL). However, when genomic RNA in STM/UTM was tested, Fusion® yielded a lower LLOD (130 copies/mL) than Aptima® (228 copies/mL) (Fig 5A and 5B).
LLOD for the Hologic Panther® SARS-CoV-2 assays determined by 5 independent measurements, performed over 5 days in duplicates (n = 10 for each datapoint). Blue: Aptima® results. Orange: Fusion® results. (A) Heat Inactivated Virus target in STM/UTM (B) Genomic RNA target in STM/UTM. The probit (predicted proportion of replicates positive) versus the SARS-CoV-2 RNA concentration identified.
False positive (FP) results
The loss of detection sensitivity in pooled samples can be explained by dilution. However, several samples tested negative by neat testing, but positive when pooled. Four out of 32 samples that previously tested negative at a clinical lab, and yielded negative results by neat testing with both Hologic SARS-CoV-2 assays, tested positive when pooled (one 5:1 and three 10:1) using two-step pooled testing with the Fusion® assay (S2A Fig). None of these four negative samples tested positive by pooled testing on the Aptima® SARS-CoV-2 assay. Variable results were observed between one-step (direct) pooling and the two-step pooling processes (S2B Fig), with a single false positive result out of 25 negative samples tested. Automated pool generation showed lower incidence of false positive results for negative sample testing, with only a single false positive result by the Aptima® assay at the 10:1 pool size (S2C Fig).
Fusion® assay less susceptible to interference between samples
To quantify interference from negative clinical discards, we calculated the 95% LLOD when using heat inactivated SARS-CoV-2 virus with either SARS-CoV-2 RNA negative post residual clinical test discards or STM/UTM to generate pools. Pools generated using post residual discards had a higher LLOD with the Aptima® assay at all pool sizes except 10:1 (S3A Fig). An improvement in LLOD was also demonstrated if the Aptima® cut-off was decreased from 560 kRLU to 350 kRLU (S3C Fig). Pooling with negative discards vs STM/UTM had mixed impact on the Fusion® assay LLOD (S3B Fig). The increase in LLOD for pooled testing with the negative clinical samples indicates the presence of interference from negative clinical samples that is more problematic for the Aptima® assay. While lowering the Aptima® cut-off to 350 kRLU does not alleviate the interference caused by NP discards, lowering the Aptima® cut-off does improve its LLOD to that of Fusion® for pools sizes ≤5:1 (S3D Fig).
Discussion
Pooling samples is a useful strategy to increase surveillance capability in order to identify SARS-CoV-2 asymptomatic individuals before they cause disease outbreaks. This study distinguishes itself from other pooled testing studies [7, 15–18] by combining the following features: large number of samples and pool sizes, comparing manual and automated methods of pooling samples, and analysis of pooled sample interaction for different nucleic acid amplification chemistries.
Our study compared both manual and automated methods of pooling samples. While many SARS-CoV-2 assays are run on high throughput automated systems that can process >1000 samples / day (Hologic Panther®, Roche cobas®), manual pooled sample generation can be rate-limiting [19]. Automated pooling of test samples improved test throughput and allowed testing of three additional pool sizes in our study (3:1, 4:1 and 8:1). Establishing automated pooling capability requires an initial investment in robotic liquid handlers and software development / verification. Software development and verification costs can be reduced by using open-source pooling scripts to implement sophisticated automated pooling systems for SARS-CoV-2 surveillance testing [20]. The average time to generate 16 pooled samples (5:1) was 6–7 minutes, including time required to load and unload samples. Up to 80 pooled samples (400 source samples) could be generated using the same strategy in approximately 24 minutes. A downstream test platform which can test 1000 pooled samples per day would require ~5 hours of automated sample pool generation.
Our pooling study tested retrospectively, while many other studies relied on prospective testing of pooled samples. With prospective testing, many of the positive pools may have more than one positive sample, thus resulting in positive pools with higher viral loads than retrospective testing. A study from Wang et al. [15] that characterized the effect of pooling on the analytical sensitivity of both the Hologic Fusion® and Aptima® SARS-CoV-2 assays generated only 36 (pool size 8) and 21 (pool size 4) positive pools with a single positive sample, limiting the statistical significance of their model. Another prospective pooled testing study from Migueres et al. [17] compared pooled sample testing of saliva samples with the Hologic Fusion® and Aptima® SARS-CoV-2 assays. This study has a similar weakness to the Wang et al. study, with only 18 positive pools with a single positive sample. However, they were able to demonstrate that Aptima® assay sensitivity is comparable to Fusion® for detecting SARS-CoV-2 in saliva samples.
Newsom et. al. [16] compared NP and saliva sample testing with the Aptima® SARS-CoV-2 TMA assay and CDC 2019-NCoV2 RT-PCR assay for pooled testing of NP samples (pool size of 10), but not the Fusion® assay. They showed that reducing the Aptima® threshold from 560 kRLU to 324 kRLU was necessary to increase sensitivity and avoid false negative results for pooled testing. Our study focused on NP samples using both Aptima® and Fusion® assays, and our results also support Newsom’s findings that reducing the Aptima® threshold increases sensitivity for pooled surveillance testing without impacting specificity. Barat et al. [7] examined testing pooled saliva samples using three different RT-PCR based assays, including the Panther Fusion® SARS-CoV-2 assay, to increase participation in a voluntary screening program. They observed lower viral load in saliva vs NP samples, so they limited pool size to 5 instead of 10, which they had used with NP samples in a previous pooling study [18].
Our study demonstrated that the same test PPA between the individual and pooled sample results for the Aptima® SARS-CoV-2 assay satisfied the FDA recommended guidelines of ≥85% [21] for pool sizes up to and including 10:1. However, these PPA results were associated with low sensitivity, especially in the automated pooling study. For the randomly sampled post residual clinical test specimens received from NIDDL and WRNMMC, 14–16% of the specimens had CT >35. This proportion concurs with previous reports of >15% of first-time diagnostic specimens with CT>35 [15]. The automated pooling results had a higher proportion of moderate viral load samples (25<CT<35, 56% vs 35%) and lower proportion of high viral load samples (CT<25, 28% vs 51%) than the manual pooling results. In addition to evaluating the same test PPA as recommended by the FDA, we suggest validating the sensitivity of the assay used for pooling, demonstrated by Wunsch et al. [22] for PCR. The Fusion® SARS-CoV-2 assay had a more consistent detection sensitivity across viral load distributions for both the manual and automated pooling. However, the Fusion® assay’s PPA and sensitivity fell below 85% for pool sizes larger than 5:1 in the automated pooling scenarios. These results agree with the studies that recommend limiting pooled sample testing to pool sizes of 5:1 and align with the label’s intended use [10, 15, 23], and contrast with studies that promote testing with pool sizes larger than 5:1 [24, 25].
The FDA has granted an EUA for pooling up to 5 samples with the Hologic Aptima® SARS-CoV-2 assay (FDA EUA200734). The Fusion® SARS-CoV-2 assay demonstrated higher sensitivity and lower LLOD than the Aptima® assay. However, lowering the cut-off threshold of the Aptima® SARS-CoV-2 assay from 560 kRLU to 350 kRLU for pooled surveillance testing would improve its sensitivity and LLOD to that of Fusion®. Specimens with higher CT >25 (moderate and low viremia) are most impacted by this shift in the cut-off value. Other studies that examined pooled surveillance testing with the Aptima® SARS-CoV-2 assay recommended thresholds of 350 kRLU [15] or 324 kRLU [16]. However, lowering the cut-off value without taking kinetics into account could have other impacts to Aptima® assay performance; therefore, further assessment would be required.
The expected CT shift for a 5:1 dilution is 2.32; the Fusion® SARS-CoV-2 assay showed a shift of 1.92 with manual pooling and 2.41 with automated pooling. The automated pooling shift was closer to the expected value, perhaps reflecting more accurate pipetting than with manual pooling. A similar result for the 10:1 dilution where the automated pooling CT shift was closer to the expected value (3.30) was also observed.
Discrepancies between sensitivity were noted for the Hologic Panther® assays, especially for the automated pooling study. The automated pooling study was conducted 6 months after the manual pooling study and used clinical discard specimens from WRNMMC instead of NIDDL. While some discrepancies are to be expected, the lower proportion of high viral load samples for the automated pool generation tests (28% vs 51% for the manual pool generation) reduced the calculated sensitivity of Aptima® more than Fusion®. Since all specimens for both manual and automated pooling were collected in the same media type (VTM), albeit sourced from various vendors, the test performance was not likely affected by collection media.
This study may have been impacted by a few limitations. We were unable to confirm the false positive results with secondary testing due to insufficient specimen volumes. The specimens were freeze-thawed once, which could have impacted the results of one assay more than the other. The samples used for the manual pooling study were stored for 1–6 months longer than those used in the automated pooling study. We cannot make a direct comparison of CT shifts due to different storage durations because the samples for manual pooling were initially tested with either the CDC RT-PCR for SARS-CoV-2 assay or the ThermoFisher TaqPath™ COVID-19 Combo assay, while the samples for automated pooling were initially tested with the Roche cobas® SARS-CoV-2 assay.
Published LLODs from other studies range from 83–288 cp/mL [26, 27] for Aptima® and from 62.5–125 cp/mL [28, 29] for Fusion®. Our analysis of LLOD for the Hologic SARS-CoV-2 assays (55–228 cp/mL for Aptima® and 108–130 cp/mL for Fusion®, Fig 5) agree with published ranges.
Preliminary studies show that the FP rate for SARS-CoV-2 RT-PCR assays is between 0.8%-4.0% [30, 31]. Currently, there are no requirements nor recommendations for specificity analysis for pooled testing. We found FP test results were less common with automated pool generation and speculate it is due to decreased pipetting errors and reduced contamination risks. Further performance assessments are required to establish specificity of pooled testing for Hologic Panther® assays.
While automated pooling offers many advantages, the high up-front and recurring costs required to establish and maintain an automated pooling system may be prohibitive for low resource settings.
Conclusion
Surveillance testing of SARS-CoV-2 infections in U.S. military personnel requires high throughput testing platform capacity linked to high throughput pooling methods. We showed that TMA assays can be a viable alternative to RT-PCR assays for detection of SARS-CoV-2 in pooling schema. While pooled testing is a simple method to increase throughput for infection surveillance, there are trade-offs between throughput and detection sensitivity that depend not only upon infection prevalence, but platform chemistry. Linking automated sample process with high throughput test capability for large scale surveillance testing provides an approach which can be readily pivoted to survey for new emerging or reemerging pathogens.
Supporting information
S1 Table. Characteristics of specimens used for this study.
https://doi.org/10.1371/journal.pone.0276729.s001
(DOCX)
S2 Table. Characteristics of the SARS-CoV-2 assays.
https://doi.org/10.1371/journal.pone.0276729.s002
(DOCX)
S3 Table. Concentrations of heat inactivated SARS-CoV-2 virus used to generate concentrations to measure LLOD.
https://doi.org/10.1371/journal.pone.0276729.s003
(DOCX)
S1 Fig. Pooled testing schema for testing with Hologic Panther Aptima® and Fusion® SARS-CoV-2 assays.
(A) In the two-step process, the negative sample pools were generated first by mixing 4 or 9 samples in equal volume for the 5:1 and 10:1 pooled testing, respectively. From each pool, 400 μL (5:1) or 450 μL (10:1) was transferred into Hologic Panther® SLT tubes, into which 100 μL (5:1) or 50 μL (10:1) of the uniquely identified positive sample was added to obtain the required testing volume of 500 μL for the Hologic Panther® assays. (B) In the one-step process, the negative and positive samples were added directly into the SLT tubes in equal volume.
https://doi.org/10.1371/journal.pone.0276729.s004
(PNG)
S2 Fig. Negative sample testing.
(A) Manual, two-step process used for pooled testing of 32 negative clinical discard specimens. False positive CT results for Fusion® shown (no FPs for Aptima®). (B) Comparison of direct pooling vs 2-step. (C) Automated generation of 200 pools (40 pools of each size) using negative specimens. The manufacturer’s Aptima® cut-off (560 kRLU) and proposed cut-off (350 kRLU) shown.
https://doi.org/10.1371/journal.pone.0276729.s005
(PNG)
S3 Fig. Pooled testing with negative clinical discards is associated with higher interference than STM/UTM.
95% LLOD shown for testing with (A) Aptima® (B) Fusion® and (C) Aptima® using a lower 350 kRLU threshold. (D) Comparison LLOD of Aptima® vs Fusion® for Negative Discards.
https://doi.org/10.1371/journal.pone.0276729.s006
(PNG)
Acknowledgments
We thank Ashley Nenninger and Abby Gajewski-Schwoch from Hologic for additional insights into the Hologic SARS-CoV-2 assays. We thank Mr. William MacTurk for helping with sample management and testing. We thank Ms. Angel Etubom for assisting with specimen transfer. We also thank the NIDDL team, especially Ms. Emna Jegham and Mr. Andrew Dudzik, for their assistance with sorting the clinical specimen discards. We thank the WRNMMC Molecular Microbiology team, led by Ms. Elana Wallace for collecting, de-identifying and organizing the clinical samples. We thank the service members whose discarded samples contributed to our study. We especially thank the COVID-19 response team that provided dedicated, skilled testing, and continued support.
Disclaimer: M.A.S., M.P.S., and Y.S.J. are U.S. Naval Officers; N.D.C. is a U.S. Army Officer; S.A.P, K.K.P. and L.L.J. are civilian U.S. Government employees. This work was prepared as part of their official duties. Title 17, U.S.C., §105 provides that copyright protection under this title is not available for any work of the U.S. Government. Title 17, U.S.C., §101 defines a U.S. Government work as a work prepared by a military Service member or employee of the U.S. Government as part of that person’s official duties. The views, opinions and/or findings contained in this report are those of the author(s) and should not be construed as an official Department of the Army, Department of the Navy, DoD, HJF or U.S. Government position, policy or decision unless so designated by other documentation.
References
- 1. Reusken CBEM Broberg EK, Haagmans B, Meijer A, Corman VM, Papa A, et al. Laboratory readiness and response for novel coronavirus (2019-nCoV) in expert laboratories in 30 EU/EEA countries, January 2020. Euro Surveill. 2020;25(6):2000082. Epub 02/11. pmid:32046815.
- 2. Schuchat A. Public Health Response to the Initiation and Spread of Pandemic COVID-19 in the United States, February 24-April 21, 2020. MMWR Morbidity and mortality weekly report. 2020;69(18):551–6. Epub 2020/05/08. pmid:32379733; PubMed Central PMCID: PMC7737947.
- 3. Statement on the second meeting of the International Health Regulations (2005) Emergency Committee regarding the outbreak of novel coronavirus (2019-nCoV) 2020. Available from: https://www.who.int/news/item/30-01-2020-statement-on-the-second-meeting-of-the-international-health-regulations-(2005)-emergency-committee-regarding-the-outbreak-of-novel-coronavirus-(2019-ncov).
- 4. Boyton RJ, Altmann DM. The immunology of asymptomatic SARS-CoV-2 infection: what are the key questions? Nat Rev Immunol. 2021;21(12):762–8. Epub 2021/10/21. pmid:34667307; PubMed Central PMCID: PMC8525456.
- 5. Mendoza RP, Bi C, Cheng HT, Gabutan E, Pagaspas GJ, Khan N, et al. Implementation of a pooled surveillance testing program for asymptomatic SARS-CoV-2 infections in K-12 schools and universities. EClinicalMedicine. 2021;38:101028. Epub 2021/07/27. pmid:34308321; PubMed Central PMCID: PMC8286123.
- 6. Chirico F, Nucera G, Ilesanmi O, Afolabi A, Pruc M, Szarpak L. Identifying asymptomatic cases during the mass COVID-19 vaccination campaign: insights and implications for policy makers. Future Virol. 2021. Epub 2021/12/25. pmid:34950218; PubMed Central PMCID: PMC8686840.
- 7. Barat B, Das S, De Giorgi V, Henderson DK, Kopka S, Lau AF, et al. Pooled Saliva Specimens for SARS-CoV-2 Testing. J Clin Microbiol. 2021;59(3). Epub 2020/12/03. pmid:33262219; PubMed Central PMCID: PMC8106731.
- 8. Mutesa L, Ndishimye P, Butera Y, Souopgui J, Uwineza A, Rutayisire R, et al. A pooled testing strategy for identifying SARS-CoV-2 at low prevalence. Nature. 2021;589(7841):276–80. Epub 2020/10/22. pmid:33086375.
- 9. Lohse S, Pfuhl T, Berkó-Göttel B, Rissland J, Geißler T, Gärtner B, et al. Pooling of samples for testing for SARS-CoV-2 in asymptomatic people. The Lancet Infectious Diseases. 2020;20(11):1231–2. pmid:32530425
- 10. Abdalhamid B, Bilder CR, McCutchen EL, Hinrichs SH, Koepsell SA, Iwen PC. Assessment of Specimen Pooling to Conserve SARS CoV-2 Testing Resources. Am J Clin Pathol. 2020;153(6):715–8. Epub 2020/04/19. pmid:32304208; PubMed Central PMCID: PMC7188150.
- 11. Watkins AE, Fenichel EP, Weinberger DM, Vogels CBF, Brackney DE, Casanovas-Massana A, et al. Increased SARS-CoV-2 Testing Capacity with Pooled Saliva Samples. Emerg Infect Dis. 2021;27(4). Epub 2021/03/24. pmid:33755009; PubMed Central PMCID: PMC8007323.
- 12. Yelin I, Aharony N, Tamar ES, Argoetti A, Messer E, Berenbaum D, et al. Evaluation of COVID-19 RT-qPCR Test in Multi sample Pools. Clin Infect Dis. 2020;71(16):2073–8. Epub 2020/05/03. pmid:32358960; PubMed Central PMCID: PMC7197588.
- 13. Esteves E, Mendes AK, Barros M, Figueiredo C, Andrade J, Capelo J, et al. Population wide testing pooling strategy for SARS-CoV-2 detection using saliva. PLoS One. 2022;17(1):e0263033. Epub 2022/01/29. pmid:35089942; PubMed Central PMCID: PMC8797214.
- 14. Kamisango K, Kamogawa C, Sumi M, Goto S, Hirao A, Gonzales F, et al. Quantitative detection of hepatitis B virus by transcription-mediated amplification and hybridization protection assay. J Clin Microbiol. 1999;37(2):310–4. Epub 1999/01/16. pmid:9889209; PubMed Central PMCID: PMC84293.
- 15. Wang H, Hogan CA, Miller JA, Sahoo MK, Huang C, Mfuh KO, et al. Performance of Nucleic Acid Amplification Tests for Detection of Severe Acute Respiratory Syndrome Coronavirus 2 in Prospectively Pooled Specimens. Emerg Infect Dis. 2021;27(1). Epub 2020/11/14. pmid:33183494; PubMed Central PMCID: PMC7774575.
- 16. Newsom K, Zhang Y, Chamala S, Martinez K, Clare-Salzler M, Starostik P. The Hologic Aptima SARS-CoV-2 assay enables high ratio pooling saving reagents and improving turnaround time. J Clin Lab Anal. 2021;35(9):e23888. Epub 2021/07/03. pmid:34213803; PubMed Central PMCID: PMC8418467.
- 17. Migueres M, Vellas C, Abravanel F, Da Silva I, Dimeglio C, Ferrer V, et al. Testing individual and pooled saliva samples for sars-cov-2 nucleic acid: a prospective study. Diagn Microbiol Infect Dis. 2021;101(3):115478. Epub 2021/08/08. pmid:34364098; PubMed Central PMCID: PMC8279932.
- 18. Das S, Lau AF, Youn JH, Khil PP, Zelazny AM, Frank KM. Pooled Testing for Surveillance of SARS-CoV-2 in Asymptomatic Individuals. J Clin Virol. 2020;132:104619. Epub 2020/09/14. pmid:32920431; PubMed Central PMCID: PMC7470739.
- 19. Chan CW, Kwon S, Matushek SM, Ciaglia C, Bethel C, Beavis KG. Implementation of a Sample Pooling Strategy for the Direct Detection of SARS-CoV-2 by Real-Time Polymerase Chain Reaction During the COVID-19 Pandemic. Am J Clin Pathol. 2021;156(1):15–23. Epub 2021/05/13. pmid:33978164; PubMed Central PMCID: PMC8136033.
- 20. Singh L, Anyaneji UJ, Ndifon W, Turok N, Mattison SA, Lessells R, et al. Implementation of an efficient SARS-CoV-2 specimen pooling strategy for high throughput diagnostic testing. Sci Rep. 2021;11(1):17793. Epub 2021/09/09. pmid:34493744; PubMed Central PMCID: PMC8423848.
- 21. Pooled Sample Testing and Screening Testing for COVID-19. U.S Food and Drug Administration 2020. Available from: https://www.fda.gov/medical-devices/coronavirus-covid-19-and-medical-devices/pooled-sample-testing-and-screening-testing-covid-19.
- 22. Wunsch M, Aschemeier D, Heger E, Ehrentraut D, Kruger J, Hufbauer M, et al. Safe and effective pool testing for SARS-CoV-2 detection. J Clin Virol. 2021;145:105018. Epub 2021/11/15. pmid:34775143; PubMed Central PMCID: PMC8552800.
- 23. Herrera LA, Hidalgo-Miranda A, Reynoso-Noveron N, Meneses-Garcia AA, Mendoza-Vargas A, Reyes-Grajeda JP, et al. Saliva is a reliable and accessible source for the detection of SARS-CoV-2. Int J Infect Dis. 2021. Epub 2021/02/14. pmid:33581365; PubMed Central PMCID: PMC7876483.
- 24. Ben-Ami R, Klochendler A, Seidel M, Sido T, Gurel-Gurevich O, Yassour M, et al. Large-scale implementation of pooled RNA extraction and RT-PCR for SARS-CoV-2 detection. Clin Microbiol Infect. 2020;26(9):1248–53. Epub 2020/06/26. pmid:32585353; PubMed Central PMCID: PMC7308776.
- 25. Zilinskas J, Lancinskas A, Guarracino MR. Pooled testing with replication as a mass testing strategy for the COVID-19 pandemics. Sci Rep. 2021;11(1):3459. Epub 2021/02/12. pmid:33568738; PubMed Central PMCID: PMC7876015.
- 26. Pham J, Meyer S, Nguyen C, Williams A, Hunsicker M, McHardy I, et al. Performance Characteristics of a High-Throughput Automated Transcription-Mediated Amplification Test for SARS-CoV-2 Detection. J Clin Microbiol. 2020;58(10). Epub 2020/07/31. pmid:32727828; PubMed Central PMCID: PMC7512162.
- 27. Cordes AK, Rehrauer WM, Accola MA, Wolk B, Hilfrich B, Heim A. Fully automated detection and differentiation of pandemic and endemic coronaviruses (NL63, 229E, HKU1, OC43 and SARS-CoV-2) on the hologic panther fusion. J Med Virol. 2021;93(7):4438–45. Epub 2020/12/23. pmid:33350484.
- 28. Smith E, Zhen W, Manji R, Schron D, Duong S, Berry GJ. Analytical and Clinical Comparison of Three Nucleic Acid Amplification Tests for SARS-CoV-2 Detection. J Clin Microbiol. 2020;58(9). Epub 2020/06/24. pmid:32571894; PubMed Central PMCID: PMC7448672.
- 29. Suh IB, Lim J, Kim HS, Rhim G, Kim H, Kim H, et al. Development and Evaluation of AccuPower COVID-19 Multiplex Real-Time RT-PCR Kit and AccuPower SARS-CoV-2 Multiplex Real-Time RT-PCR Kit for SARS-CoV-2 Detection in Sputum, NPS/OPS, Saliva and Pooled Samples. PLoS One. 2022;17(2):e0263341. Epub 2022/02/11. pmid:35143538; PubMed Central PMCID: PMC8830688.
- 30. Surkova E, Nikolayevskyy V, Drobniewski F. False-positive COVID-19 results: hidden problems and costs. The Lancet Respiratory Medicine. 2020;8(12):1167–8. pmid:33007240
- 31. Cohen AN, Kessel B, Milgroom MG. Diagnosing SARS-CoV-2 infection: the danger of over-reliance on positive test results. medRxiv. 2020;04(26).