https://orcid.org/0000-0002-0631-5834
Peer Review History
| Original SubmissionOctober 5, 2021 |
|---|
|
PONE-D-21-30196Real-world utilization of SARS-CoV-2 serological testing in RNA positive patients across the United StatesPLOS ONE Dear Dr. Rodriguez-Watson, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Dec 30 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript:
If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Jue Liu, Ph.D. Academic Editor PLOS ONE Journal Requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and 2. We note that Figure 1 in your submission contain map images which may be copyrighted. All PLOS content is published under the Creative Commons Attribution License (CC BY 4.0), which means that the manuscript, images, and Supporting Information files will be freely available online, and any third party is permitted to access, download, copy, distribute, and use these materials in any way, even commercially, with proper attribution. For these reasons, we cannot publish previously copyrighted maps or satellite images created using proprietary data, such as Google software (Google Maps, Street View, and Earth). For more information, see our copyright guidelines: http://journals.plos.org/plosone/s/licenses-and-copyright. We require you to either (1) present written permission from the copyright holder to publish these figures specifically under the CC BY 4.0 license, or (2) remove the figures from your submission: 1. You may seek permission from the original copyright holder of Figure 1 to publish the content specifically under the CC BY 4.0 license. We recommend that you contact the original copyright holder with the Content Permission Form (http://journals.plos.org/plosone/s/file?id=7c09/content-permission-form.pdf) and the following text: “I request permission for the open-access journal PLOS ONE to publish XXX under the Creative Commons Attribution License (CCAL) CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). Please be aware that this license allows unrestricted use and distribution, even commercially, by third parties. Please reply and provide explicit written permission to publish XXX under a CC BY license and complete the attached form.” Please upload the completed Content Permission Form or other proof of granted permissions as an "Other" file with your submission. In the figure caption of the copyrighted figure, please include the following text: “Reprinted from [ref] under a CC BY license, with permission from [name of publisher], original copyright [original copyright year].” 2. If you are unable to obtain permission from the original copyright holder to publish these figures under the CC BY 4.0 license or if the copyright holder’s requirements are incompatible with the CC BY 4.0 license, please either i) remove the figure or ii) supply a replacement figure that complies with the CC BY 4.0 license. Please check copyright information on all replacement figures and update the figure caption with source information. If applicable, please specify in the figure caption text when a figure is similar but not identical to the original image and is therefore for illustrative purposes only. The following resources for replacing copyrighted map figures may be helpful: USGS National Map Viewer (public domain): http://viewer.nationalmap.gov/viewer/ The Gateway to Astronaut Photography of Earth (public domain): http://eol.jsc.nasa.gov/sseop/clickmap/ Maps at the CIA (public domain): https://www.cia.gov/library/publications/the-world-factbook/index.html and https://www.cia.gov/library/publications/cia-maps-publications/index.html NASA Earth Observatory (public domain): http://earthobservatory.nasa.gov/ Landsat: http://landsat.visibleearth.nasa.gov/ USGS EROS (Earth Resources Observatory and Science (EROS) Center) (public domain): http://eros.usgs.gov/# Natural Earth (public domain): http://www.naturalearthdata.com/ [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Partly Reviewer #3: Yes ********** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes Reviewer #3: Yes ********** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes Reviewer #3: Yes ********** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes Reviewer #3: Yes ********** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: The authors conducted a review of electronic health records and/or claims data across six major health systems in the United States of America to assess the real-world utilisation of serological testing for SARS-CoV-2 in patients who had tested positive for SARS-CoV-2 via RNA previously. They identified 930,669 individuals and noted that approximately 15% of them underwent serological testing within 14-days of positive diagnosis via RNA. Unfortunately the study was limited by its retrospective nature and its reliance on EHR and claims data and as such a fair amount of data is missing including race/ethnicity data, molecular and RNA test type and symptomatology. However, as the intention of this manuscript is to characterise the utilisation of serological testing for SARS-CoV-2 in a real world setting, this missing data does not compromise the integrity of the study but in fact highlights shortcomings in the health system, in terms of critical information capturing such as demographic and clinical data. Furthermore, the manuscript also gives insights into health practices during the pandemic including non-adherence to clinical guidelines related to SARS-CoV-2 testing and the use of serological assay. This information is important as it can inform future practice and highlight the need to follow evidence based guidance during the pandemic. Notably the data is heavily skewed towards Providers B and F (> 66% of the cohort) but as the data is presented individually this is clear. The manuscript may benefit from the inclusion of a "total" column in all tables although this may be difficult to express the data due to the large amount of missing information. Thank you for submitting this manuscript for review. Reviewer #2: This is an impressive dataset collected from multiple systems addressing principles of inter-operability, population coverage and linkage of laboratory results with detailed clinical metadata to address policy or public health needs. The main concern is whether the objectives set have made sufficient progress to justify publication at this point. It also would benefit from more detailed explanation of the research questions behind the objectives set (presumably by the FDA) to support presentation of the progress made in the form of a research publication, rather than alternatively as a published interim project progress report. It also seems aligned to the requirements of the US approach to FDA diagnostics approvals, implementation and guidance. This makes it a bit hard to clearly see the specific research questions as opposed to providing data to support decision making within the US diagnostics regulatory framework. The comments below expand on these concerns which are raised from perspective of a non-US infectious diseases clinician involved in SARS-CoV-2 testing including serological tests. Consequently some concerns may be easily addressed or reflect misunderstanding due to lack of familiarity with the US setting and how this work fits in. Abstract - conclusion: how exactly does this methodological approach actually address the question of determining performance data or linking with manufacturer and clinical data. It does not seem appropriate to say the need for more efficient testing strategies without presenting the actual strategies and rationale for testing in these different settings at that time. Given the dynamic nature of this disease there is potential for findings of a study like this to not be relevant by the time a follow up study is done or recommendations are made for the future. For example, in the study period serology testing was sometimes performed to help with diagnosis of what was a new condition. Antibodies appear around day 10 and so without vaccination or potential for prior infection, it was not inappropriate to perform serology to help with diagnosis. Duration of symptoms was an important clinical measure not collected in this study. In addition, patients now get serology testing on admission to determine which SARS-CoV-2 RNA-PCT positive patients are antibody negative and might require Regeneron or similar Finally, now that vaccination is widespread serology testing is used more to assess immunity (not prevalence/incidence of disease) or as a universal hospital admission test to inform infection control decisions (ie antibodies +/- history of vaccination = protected and less likely to acquire SARS-CoV-2). Line 69 serology is not only done to determine prevalence estimates Line 73 Please explain for those not familiar with US regulatory processes what the relevance of 510(k) is and what the link is between approvals and this collection of real world evidence Line 81 -83. Interoperability and linkage of tests with metadata from EHR impacts on all of medicine but what are the advantages of this approach here for determining test performance and linking with clinical symptoms. Please explain the benefits compared with studies conducted at representative institutions / different clinical settings where local laboratory and clinical data can be collected alongside guidelines and rationale. Realise the national approach will get there but it is a long and complex road to get to the granularity required to meet stated objectives. Line 93. The FDA asked for this consortium to be established with specific purpose. The consortium has a broad range of experts and links diverse providers. Line 100 sets 3 different objectives. How does the meeting of those objectives link into a research question and publication? Line 109: How did these organisations came together and are representative of the organisations involved in serology. They presumably were not randomly selected. The authors do say heavy representation from certain states Line 126. March to September was very early on in the pandemic. First wave. Diagnostics and guidelines have improved significantly. All tested parameters are very different now, hence making generalisability comments, even for the US, seems tenuous for today. From line 128: it is hard to understand the locations and factors underpinning access to diagnostic tests from outside the US system to interpret results and assess potential biases and confounders Line 251; Duration of symptoms is an important measure in determining whether to perform serology testing alongside PCR testing at time of first presentation Line 278: What is is the clinical purpose of doing routine follow up serology testing in a PCR confirmed SARS-CoV-2 patient. Line 317. Don’t agree with this statement. As mentioned above there have been many different reasons for doing antibody testing. Without incorporating perspectives of what is happening on the ground as things change fast, it is inappropriate to link with unreferenced and likely now updated “clinical guidance” Line 327: data of PCR test is not a good proxy for date of symptom onset. PCR testing in many studies is reported to be done between 5 and 15 days post onset of symptoms when patients present to healthcare. Accept this may be different in the US Reviewer #3: 1) Real-world utilization of SARS-CoV-2 serological testing in RNA positive patients across the United States A very frustrating, slightly annoying but not unexpected finding that there is no coherent data collection between providers. That said the major quantitative finding of the study appears to be of the 930,669 individuals with positive RNA for SARS-CoV-2 identified, 15% had serotesting <14 days from the RNA positive test. Positivity at 14 days can be as high as 95% for hospitalized patients corresponding to large viral loads and arguably slow immune response. The study was surprisingly predominantly female despite the prevalence and increased risk in males over 55. Why in such a large cohort was this bias observed? A solid set of data analysis but I was left with a real feeling of ‘so what?’. The study was not wrong, appears competently performed but lacked truly useful data because this is not collected by the partners. No inter-platform analysis, no identification of the antibody test performed, no analysis of false negatives from the PCR test (80%), no analysis of false positive from PCR test (4%), no analysis of estimate of recording errors (1 per 1000 entries). ********** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: No Reviewer #3: No [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.
|
| Revision 1 |
|
PONE-D-21-30196R1Real-world utilization of SARS-CoV-2 serological testing in RNA positive patients across the United StatesPLOS ONE Dear Dr. Rodriguez-Watson, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Jun 11 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript:
If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Jue Liu, Ph.D. Academic Editor PLOS ONE Journal Requirements: Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation. Reviewer #1: (No Response) Reviewer #2: All comments have been addressed ********** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ********** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: I Don't Know ********** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ********** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ********** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: Thank you for the changes made following review. The manuscript remains relevant in that it gives insights into health practices and the use of serological testing during the early days of the pandemic. Despite rapid changes in the pandemic since the date of initial submission this information may still inform future practice with COVID and other viral infections to come. Two minor recommendations: Tables pages 10 - 15: Thank you for adding the total column to the table. However, the column shows a total for both "Yes" and "No" for serological testing status which adds little information. This reviewer feels it will be better to show a Total column for "Yes" and a total column for "No" in order to compare the broad outcomes of serological testing in each group. Table, page 23: "Any Chargemaster or Medical Claim" Please clarify in the footnotes what "Chargemaster" is. As a non-US reader/reviewer I am not sure of the relevance of this. Thank you for resubmitting your work. Reviewer #2: (No Response) ********** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: Yes: Jonathan Edgeworth [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.
|
| Revision 2 |
|
Real-world utilization of SARS-CoV-2 serological testing in RNA positive patients across the United States PONE-D-21-30196R2 Dear Dr. Rodriguez-Watson, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, AbdulAzeez Adeyemi Anjorin, Ph.D. Academic Editor PLOS ONE Additional Editor Comments (optional): Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation. Reviewer #1: All comments have been addressed ********** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes ********** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes ********** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes ********** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes ********** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: Thank you for the clarification and the corrections. I believe it is now acceptable for publication. ********** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No ********** |
| Formally Accepted |
|
PONE-D-21-30196R2 Real-world utilization of SARS-CoV-2 serological testing in RNA positive patients across the United States Dear Dr. Rodriguez-Watson: I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department. If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org. If we can help with anything else, please email us at plosone@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. AbdulAzeez Adeyemi Anjorin Academic Editor PLOS ONE |
Open letter on the publication of peer review reports
PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.
We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.
Learn more at ASAPbio .