PONE-D-20-36825

Saliva as a testing specimen with or without pooling for SARS-CoV-2 detection by multiplex RT-PCR test

PLOS ONE

Dear Dr. Sha,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Jan 30 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Ruslan Kalendar, PhD

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Thank you for stating the following financial disclosure:

"This study not involve any funded"

At this time, please address the following queries:

1. Please clarify the sources of funding (financial or material support) for your study. List the grants or organizations that supported your study, including funding received from your institution.
2. State what role the funders took in the study. If the funders had no role in your study, please state: “The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.”
3. If any authors received a salary from any of your funders, please state which authors and which funders.
4. If you did not receive any funding for this study, please state: “The authors received no specific funding for this work.”

Please include your amended statements within your cover letter; we will change the online submission form on your behalf.

3. Thank you for stating the following in the Competing Interests section:

"no any competing interests"

We note that one or more of the authors are employed by a commercial company: 1DiaCarta Inc. 2600 Hilltop Dr. Richmond, CA 94806.

(1) Please provide an amended Funding Statement declaring this commercial affiliation, as well as a statement regarding the Role of Funders in your study. If the funding organization did not play a role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript and only provided financial support in the form of authors' salaries and/or research materials, please review your statements relating to the author contributions, and ensure you have specifically and accurately indicated the role(s) that these authors had in your study. You can update author roles in the Author Contributions section of the online submission form.

Please also include the following statement within your amended Funding Statement.

“The funder provided support in the form of salaries for authors [insert relevant initials], but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The specific roles of these authors are articulated in the ‘author contributions’ section.”

If your commercial affiliation did play a role in your study, please state and explain this role within your updated Funding Statement.

(2) Please also provide an updated Competing Interests Statement declaring this commercial affiliation along with any other relevant declarations relating to employment, consultancy, patents, products in development, or marketed products, etc.  

Within your Competing Interests Statement, please confirm that this commercial affiliation does not alter your adherence to all PLOS ONE policies on sharing data and materials by including the following statement: "This does not alter our adherence to  PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests) . If this adherence statement is not accurate and  there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared.

Please include both an updated Funding Statement and Competing Interests Statement in your cover letter. We will change the online submission form on your behalf.

Please know it is PLOS ONE policy for corresponding authors to declare, on behalf of all authors, all potential competing interests for the purposes of transparency. PLOS defines a competing interest as anything that interferes with, or could reasonably be perceived as interfering with, the full and objective presentation, peer review, editorial decision-making, or publication of research or non-research articles submitted to one of the journals. Competing interests can be financial or non-financial, professional, or personal. Competing interests can arise in relationship to an organization or another person. Please follow this link to our website for more details on competing interests: http://journals.plos.org/plosone/s/competing-interests

4. We note that Table 3A, 3B, and 3C have been previously published in https://www.fda.gov/media/136809/download. Thus, we require specific consent from the copyright holder to publish these tables in PLOS ONE, under the CC BY 4.0 license. To seek permission from the copyright owner to publish these tables under the Creative Commons Attribution License (CCAL), CC BY 4.0, please contact them with the following text and PLOS ONE Request for Permission form (http://journals.plos.org/plosone/s/file?id=7c09/content-permission-form.pdf):

“I request permission for the open-access journal PLOS ONE to publish XXX under the Creative Commons Attribution License (CCAL) CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). Please be aware that this license allows unrestricted use and distribution, even commercially, by third parties. Please reply and provide explicit written permission to publish XXX under a CC BY license.”

Please upload the granted permission to the manuscript as a Supporting Information file. In the caption of the copyrighted tables, please include the following text: “Republished from [ref] under a CC BY license, with permission from [name of publisher], original copyright [original copyright year].

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

Reviewer #1: Partly

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: No

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

**********

5. Review Comments to the Author

Reviewer #1: The manuscript entitled “Saliva as a testing specimen with or without pooling for SARS-CoV-2 detection by multiplex RT-PCR test.” by Sun and colleagues described their evaluation of a FDA-EUA commercial assay for SARS-CoV-2 detection by multiplex RT-PCR using saliva as a specimen type. Also, several papers have already been published on detection of SARS-CoV-2 in saliva, this particular kit was not evaluated for saliva specimens. The manuscript is generally well written, but it has several major concerns with data presentation and conclusions made:

Major concerns:

In the method section, the authors described how the have designed the primers and probes for the assay although their sequences were not provided. Also, it appears Taqpath mastermix was used for RT-qPCR. It is not clear if these are part of the components/methods for the commercial kit they have described?

Figure 2: Tt is not clear what is the relevance of the ROC curve? ROC is used to determine cut-off or threshold that gives best sensitivity and specificity. How the ROC curve and AUC was calculated? The current figure suggests decreasing specificity with increasing sensitivity! Also the AUC does not seem be not correct!

Table 1 and 3: Data analyzed and presented in such a manner which is inappropriate for evaluation of performance characteristics of a clinical assay. The limit of detection in saliva should be empirically determined with proper statistics. The authors may refer to the guidelines in this review article: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2901657/

Also presenting data in Table 3 as PPA and NPA, as part of clinical evaluation, is not correct because these are all spiked samples with known quantity of the analyte, within the analytical range, they are all expected to be detected anyway. However, these results may be presented to determine inhibition/interference from saliva specimens (if different saliva samples were utilized), and may also be used to determine intra-assay and inter-assay variability using PCR CT values.

Table 2: Why RP is negative in most samples? How many samples for each target was tested? How SD and CI were calculated?

Table 6: unnecessary repeat in different columns - detected vs not detected

CT significantly weaker in the positive samples by Quantivirus compared to the reference method but there were 3 extra positive results by Quantivirus needs further verification to ensure that they were not false positive results. Also, it is not clear how the results of prospective evaluation with only 24 samples in this case compares with their retrospective data in Table 4 in terms of sensitivity and specificity?

Table 8 is confusing. Poos better expressed as the pool of 1 pos + 5 neg specimen. How the specimens were tested. How does the CT values compare between single vs pooled runs? Because pooled runs were made using pre-selected samples, results are better to express as % agreement instead of sensitivity/specificity/PPV/NPV

Minor concerns:

Page 13 last paragraph: pooled health sample – check grammar in all the places this term was used.

Too many tables with detailed raw data. Show data summary instead and add supplemental tables if necessary.

Reviewer #2: 

The authors demonstrate that saliva specimens can be reliably used for SARSCoV-2 detection, and saliva-based large-scale population screening for COVID-19 with or without pooling is feasible. I think this is important to help many people who have problems, to do nose picking and get answers easily and safely, overcoming the drawbacks of nasopharyngeal (NPS) or oropharyngeal swabs (OPS) swabs. According to the statistics produced in this paper, the clinical performance of saliva-based tests is comparable to that of NPS-based clinical tests.

Reviewer #3: 

The manuscript describes the validation of both non-pooled and pooled saliva samples compared to the gold std of NP samples. The data is very good and supports the conclusions made. This is an important manuscript given the relative ease of sample collection, the amenability to high throughput and pooling and the accuracy of the results and should be published.

i had only one question re the diluting of the saliva samples: i thought the denominator (2nd #) was supposed to indicate the # of total dilutions rather than the # of diluent (healthy) samples as in titers for serology. Thus, the first ratio should be 1:6 and the 2nd a 1:12 - ? a minor point.

Reviewer #4: 

This is a nice study to extend the use of a validated SARS-Cov-2 multiplex RT-PCR assay for detection in saliva. Overall, it is well written, thorough, and the appropriate statistical analyses have been included.

MAJOR Concerns:

1. Financial Disclosure: Clearly this study was supported (people, reagents, instruments, etc...) and funded by DiaCarta. This should be included on the financial disclosure.

2. Competing Interests: Nine out of the twelve authors are associated, i.e. employed, by DiaCarta, LLC. who is the manufacturer/developer of this QuantiVirus kit. As stated in the Plos ONE Competing Interests policy, employment is considered a financial competing interest and authors should provide details on the relationship to the funder (i.e. employment) and a "Description of funder’s role in the study design; collection, analysis, and interpretation of data; writing of the paper; and/or decision to submit for publication."

The fact that these items were not included, and even submitted with the manuscript with statements of... "This study not involve any funded" and "no any competing interests" is the biggest concern I have regarding publication.

MINOR Concerns and Comments:

General:

1. Please double check to ensure all acronyms are spelled out the first time they are used. MGI, CLC, UNG, ROC, etc..

Introduction:

1. Some description of the saliva collection devices and process, in general as well as from other studies would be useful. Is the collection from stimulated saliva or passive drool? Are comparable volumes collected?

2. Reference is needed near the bottom of page 4 following... "Other researchers also reported that SARS-CoV-2 was detected in 91.7% (n=11) of the initial saliva specimens from confirmed COVID-19 patients. (REF)"

Methods:

1. All of the methods generally lack detail, particularly with respect to the sample pooling and analytical sensitivity. Typically the analytical sensitivity involves a serial dilution of a known concentration or spiking of negative samples with a known amount of viral RNA at decreasing concentrations. Then the LOD is established as the 0 copies/mL + 3 standard deviations. If this or another method was used it needs to be described clearly and with sufficient detail that someone/anyone could repeat it and obtain the same result.

Results/Discussion:

1. How does the analytical sensitivity of 100-200 copies/mL compare to other test kits (multi-plex and non)? It may be helpful to include an example graph of the analytical sensitivity data rather than just a summary table.

2. On page 12, last sentence in section titled "Clinical evaluation of paired NPS and saliva samples" you should remove the plural (s) from "samples types"

3. On page 13, in the section titled "Population screening using saliva samples" the dates of collection do not correspond to what is written in the Methods section...was this population screening included in the IRB exemption? The way it is written sounds like the samples were collected specifically for this study and not excess/otherwise discarded. Additional clarification may be useful.

4. On page 13, last paragraph third line from the bottom "health" should be "healthy" in two instances...this also occurs in Table 8 left hand column.

5. Additional clarity is needed to describe the pooled samples (back in the Methods section would be best) because it is unclear if 77 distinct pooled samples from unique donors (1 patient + 5 healthy individuals) were created and used. Also, it is not clear why there was a different number of 1+5 pooled samples vs. 1+11 pooled samples were analyzed (N=77 vs N=49).

6. Table 8, what is the column labeled "Sample Screen(N)" there doesn't seem to be any mention or reference to that anywhere else in the manuscript, but maybe I missed it.

7. First line of your Discussion...I would add "We have developed and validated a multiplex RT-qPCR assay for SARS-CoV-2 detection in saliva..." so that you can emphasize the distinction of this work from previous assay validation in NPS and its significance.

8. Page 15, third line from the top...I would consider re-wording "On the other hand" phrase as it sounds negative and the statement is a quite positive one regarding the lack of inhibition from saliva.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

Reviewer #1: No

Reviewer #2: No

Reviewer #3: Yes: Melissa Kennedy

Reviewer #4: No

Saliva as a testing specimen with or without pooling for SARS-CoV-2 detection by multiplex RT-PCR test

PONE-D-20-36825R1

Dear Dr. Sha,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Ruslan Kalendar, PhD

Academic Editor

PLOS ONE