PONE-D-20-31021

Performance of SARS-CoV-2 Serology tests: Are they good enough?

PLOS ONE

Dear Dr. Piec,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

==============================

ACADEMIC EDITOR: Please review comments made by the reviewers and provide point by point response in your revised manuscript.

==============================

Please submit your revised manuscript by due date. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Muhammad Adrish

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1) Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2)  Thank you for stating the following in the Acknowledgments Section of your manuscript:

[We finally wish to thank the Norfolk, Suffolk, Essex and Hertfordshire Freemasons for their generous financial support to purchase the Agility system.]

We note that you have provided funding information that is not currently declared in your Funding Statement. However, funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form.

Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows:

 [The funders had no role in study design, data collection and analysis, decision to

publish, or preparation of the manuscript.]

Please include your amended statements within your cover letter; we will change the online submission form on your behalf.

3) Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Major comment: The manuscript# PONE-D-20-31021entitled “Performance of SARS-CoV-2 Serology tests: Are they good enough?” assess the performance of SARS-CoV-2 serologic tests available in clinical practices.

These methods were introduced into clinical practice quickly without an extensive validation usually required by the regulatory authorities. The IgG serologic Assays from EuroImmun, Epitope Diagnostics (EDI), Abbott Diagnostics and DiaSorin and a rapid IgG/IgM test from Healgen were evaluated for the imprecision, diagnostic sensitivity and specificity and concordance. The SARS-CoV-2 patient samples were analyzed as pre-pandemic control specimen with confirmed respiratory infections, inflammatory polyarthritis and several samples positive for thyroid stimulating immunoglobulin and post pandemic samples. Authors showed that the diagnostic sensitivity of the assays were 100% (CI: 80-100%) for Abbott, EDI and EuroImmun and 95% (CI: 73-100%) for DiaSorin at ≥14 days post infection. Only the Abbott assay had a diagnostic specificity of 100% (CI: 91-100%). The DiaSorin displayed more false negative results across all conditions tested. The imprecision for EDI and EuroImmun, was 0.02-14.0% CV while for Abbott and DiaSorin imprecision (CV) ranged from 5.2% - 8.1% and 8.2% - 9.6% respectively. Overall, concordance of the assays ranged from 76.1% to 97.9%. The point of care rapid test Healgen, showed excellent sensitivity and specificity. Authors attributed the differences in results to the use of nucleocapsid and spike proteins. The point of care device tested demonstrated adequate performance for antibody detection.

Limited sample size should be included in the limitation of the study.

The Figure legend describes Figure 1 as Box-plot even though data shown as a dot plot.

The Figure 1A reveled very little to no cross reactivity. The statistical analysis among groups show significant differences between N to RA and N to TSI. None of the values in RA and only one value in TSI groups is above the positive cutoff value. May be author should comment on meaning of such statistical significance.

In this study the DiaSorin displayed the most-false negatives and showed cross-reactivity when comparing with other serologic assays. It will important to add in the discussion the frequency of use of these tests within UK or globally specially if that data is available through health care systems or through the company.

Reviewer #2: This is a well written manuscript. COVID -19 is also a huge demanding topic. I have some minor comments.

Overall: The number of positive samples is very low (43). Though the present analysis showed a good agreement. But it may alter if the sample size increased. There are so many published paper with the information of different type of serological testing of SARS-CoV-2. Presently we need more specific decision for specific serological test.

Please add some explanation in discussion section, why the specificity of DiaSorin assay showed low.

Line 111: Please add the time period of the SARS-CoV-2 positivity; like March to May.

Reviewer #3: The study is well designed and results are clear presented.

Concerning the results higher sensitivity for point of care devices was found, compared with other publications. The absence of cross-reactivity to seasonal coronavirus and influenza A is a remarkable result. Evolution of the virus and antibodies production turns serology assays a complex and hard work.

Although Authors did not consider strengths or weaknesses of the work.

To date the samples for serology PCR day was used, but it would be better to date from onset of symptoms.

The study is small and results probably will need to be reproduced in larger studies, in different places and in systematic reviews.

Line 268 – we chose – please correct.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

Reviewer #3: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Attachments

Attachment

Submitted filename: Review PONE-D-20-31021-11-6-20.docx

Performance of SARS-CoV-2 Serology tests: Are they good enough?

PONE-D-20-31021R1

Dear Dr. Piec,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Muhammad Adrish

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

All comments have been addressed

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

Reviewer #3: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: (No Response)

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: (No Response)

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: (No Response)

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: (No Response)

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Comment: The manuscript# PONE-D-20-31021R1 entitled “Performance of SARS-CoV-2

Serology tests: Are they good enough?” assessed the performance of SARS-CoV-2 serologic tests available in clinical practices. The topic is highly relevant in the current settings. In this revised manuscript authors have addressed the comments and suggestions provided by each of the reviewers.

Reviewer #2: Author addressed all the comments properly. It is a great opportunity for me to be a reviewer of this manuscript which is related to COVID-1.

Reviewer #3: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

Reviewer #3: No