Operational analysis for COVID-19 testing: Determining the risk from asymptomatic infections

Marc Mangel

doi:10.1371/journal.pone.0281710

Peer Review History

Original SubmissionJuly 25, 2022
26 Oct 2022 Decision Letter - Rehana Naz, Editor PONE-D-22-20932Operational analysis for coronavirus testing: Positivity is not incidence, risk is not binary, and the fraction of asymptomatic infections can be determinedPLOS ONE Dear Dr. Mangel, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Dec 10 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Rehana Naz Academic Editor PLOS ONE Journal Requirements: . When submitting your revision, we need you to address these additional requirements. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. Thank you for stating the following in the Acknowledgments Section of your manuscript: "Financial support was provided by a consulting agreement with the Johns Hopkins University Applied Physics Laboratory (APL). I thank Alan Brown for inviting me, early in the pandemic, to think about operations analysis of testing for coronavirus and Matt Shaffer for his steady support during the work." We note that you have provided funding information that is not currently declared in your Funding Statement. However, funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form. Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows: "MM was supported by a consulting contract with the Johns Hopkins University Applied Physics Laboratory. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript." Please include your amended statements within your cover letter; we will change the online submission form on your behalf. 3. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice. Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Partly Reviewer #3: Yes ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: No Reviewer #3: Yes ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes Reviewer #3: Yes ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes Reviewer #3: Yes ****** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: The author has shown new applications of a method to translate surface positivity to estimate incidence rates. The author has also discussed limitations of the study. The manuscript is well written and is a significant contribution to the field. I recommend it for publication. Reviewer #2: The following points should be considered in the manuscript: 1-In the equation-6, the relationship is proportional to and how the author took the logarithm and the derivative of the two side themselves in the equation-7 and even for the far right side, the details must be explained in the manuscript. 2-In the figures-8, 9, and 10, the results give high possibilities of testing errors, of course based on the author results, however, this is not the actual case and the author must add lots of details about those results and their relation to the actual cases. 3-The title of the manuscript is a vague and only can be used in blogs rather than in scientific article and this title must be reworded. 4-There are some equations are approximated equations, especially, in thé appendices. So, the author must mention this point in the manuscript for each one. 5-There are some symbols or abbreviations used in the manuscript without defining them, so, the author must define each symbol even if the symbol is a well known. 6-It is not preferable at all using the first person pronoun 'I' as the author did in the manuscript, especially, in the abstract. 7-The reasons of choosing of the value of the (MLE) fˆ in the results of the figures must be explained in details. 8-The author in the title of the manuscript and other places of the manuscript has used the term 'coronavirus', however, the manuscript focuses on a specific type of the coronaviruses and this point must be reconsidered. 9-The computational errors of the calculations must be discussed with lots of details because the results of the fractions are not consistent with the actual case. Reviewer #3: The author summarizes some results from a prior study, where an estimator is derived for the incidence rate if the surface positivity and probabilities of false positive and false negative tests are known and the range for the value of the incidence rate (‘test rate’) is obtained. A formula for the test range is derived in the case when the surface positivity is less than the probability of a false positive. Further, the results of the earlier study are generalized when information regarding symptomatic and asymptomatic individuals is known at the time of testing, including point estimate and range for the ratio of asymptomatic to symptomatic cases. Comments: The work deals with a very relevant problem, and is well communicated. Possible concerns regarding the approach have been spelt out and satisfactorily answered in the ‘Discussion’ section. I would suggest that since the work draws from the previous reports ([29] and [30]), it may be a good idea to include the calculations from those studies in the mathematical appendix, as this will make this paper more self-contained. There may be a typo on Line 281 ****** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: No Reviewer #3: Yes ******** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0281710.r001
Revision 1
16 Dec 2022 Author Response Response to comments from the reviewers 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Partly Reviewer #3: Yes Response: In response to the specific comments of Reviewer #2, detailed below, I addressed all the points that lead to the “Partly” answer. 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: No Reviewer #3: Yes Response: The statistical analysis, both Bayesian and likelihood methods, is indeed rigorously performed. I believe that the answer from Reviewer #2 is a result of the Reviewer expecting more details of some of the analytical steps. As described below, these are now included, both in the main text ant the supplementary information (depending upon the level of detail involved). 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes Reviewer #3: Yes Response: No response needed. 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes Reviewer #3: Yes Response: No response needed. 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: The author has shown new applications of a method to translate surface positivity to estimate incidence rates. The author has also discussed limitations of the study. The manuscript is well written and is a significant contribution to the field. I recommend it for publication. Response: Thank you for these comments. Although the manuscript organization has not changed, I worked to improve and tighten the writing and, as detailed below in the response to Reviewer #2, I details of analysis at various points in the manuscript. These are indicated by the line numbers in the new version of the manuscript. Reviewer #2: The following points should be considered in the manuscript: Response: Thank you for these thoughtful comments, which helped me make the approach clearer. 1-In the equation-6, the relationship is proportional to and how the author took the logarithm and the derivative of the two side themselves in the equation-7 and even for the far right side, the details must be explained in the manuscript. Response: I removed the proportional to in Eqn 6, using equality instead and then added a sentence below it explaining that although the mathematical equation is the same, the variables play different roles. I also expanded Supplementary information S1 Brief review of the binomial distribution and binomial likelihood to include a new sub-section that contains a derivation of Eqn 7. I believe that the detailed derivation is better left in the supplementary information because it is possible to then include all of the mathematical details, taking nothing for granted on the part of the reader. That is, the mathematical tools are relative simple but used in mature ways and putting the derivation in the supplementary information allows me to be very explicit and clear about how the derivation works. 2-In the figures-8, 9, and 10, the results give high possibilities of testing errors, of course based on the author results, however, this is not the actual case and the author must add lots of details about those results and their relation to the actual cases. Response: I now cite references 22-27 to justify the choice of the true state of nature and the test errors (lines 316-319) and make reference to supplementary information S3 Sensitivity analysis when there is information on symptoms for other assumptions about the true state of nature. In addition, I added a point in the discussion (lines 403-410) to emphasize that what is developed here is a procedure to go from test information to the risk of including asymptomatic individuals in groups of different sizes, so that one can apply the procedure to their own choice of true states of nature and test errors. 3-The title of the manuscript is a vague and only can be used in blogs rather than in scientific article and this title must be reworded. Response: As the reviewer surmised, I struggled with finding the right title. The title is now “ Operational analysis for COVID-19 testing: Determining the risk from asymptomatic infections” 4-There are some equations are approximated equations, especially, in thé appendices. So, the author must mention this point in the manuscript for each one. Response: The approximated equations are explained in more detail (lines 240-246 and below Eqn S19 in the Supporting information), along with additional citation to reference 30 in which the delta method is explained in detail. 5-There are some symbols or abbreviations used in the manuscript without defining them, so, the author must define each symbol even if the symbol is a well known. Response: All symbols are now explained at the very first time they are introduced. 6-It is not preferable at all using the first person pronoun 'I' as the author did in the manuscript, especially, in the abstract. Response: I removed the first person from the abstract, but after checking with the editor handling the manuscript left first person in some places in the main text, because otherwise I would have had to start writing in passive rather than active tense. 7-The reasons of choosing of the value of the (MLE) fˆ in the results of the figures must be explained in details. Response: Additional explanations for the choice of MLE (lines 37-39) and the compatibility intervals and regions (lines 46-49, 262-264) are given. 8-The author in the title of the manuscript and other places of the manuscript has used the term 'coronavirus', however, the manuscript focuses on a specific type of the coronaviruses and this point must be reconsidered. Response: I now refer to COVID-19 in the title and in the first sentence of the manuscript write “Entering the third year of the 2019 coronavirus disease (henceforth COVID-19) 9-The computational errors of the calculations must be discussed with lots of details because the results of the fractions are not consistent with the actual case. Response: I have added discussion, particularly regarding Figure 10, concerning how we interpret the results of the calculations when they differ from the true state of nature. It is possible, however, that I misunderstand what is meant in this comment by “the actual case” Reviewer #3: The author summarizes some results from a prior study, where an estimator is derived for the incidence rate if the surface positivity and probabilities of false positive and false negative tests are known and the range for the value of the incidence rate (‘test rate’) is obtained. A formula for the test range is derived in the case when the surface positivity is less than the probability of a false positive. Further, the results of the earlier study are generalized when information regarding symptomatic and asymptomatic individuals is known at the time of testing, including point estimate and range for the ratio of asymptomatic to symptomatic cases. Comments: The work deals with a very relevant problem, and is well communicated. Possible concerns regarding the approach have been spelt out and satisfactorily answered in the ‘Discussion’ section. Response: An excellent summary. I would suggest that since the work draws from the previous reports ([29] and [30]), it may be a good idea to include the calculations from those studies in the mathematical appendix, as this will make this paper more self-contained. Response: The two previous reports are easily accessed through the links given in the list of references, so I have not followed this suggestion because it would add considerable length to the paper. There may be a typo on Line 281 Response: I have rewritten the line, because the explanation of the method is given earlier in the paper. In essence, I am generalizing Hudson’s (1971) method in which the confidence interval (I use compatibility interval) is found by moving 2 units down from the peak of the log-likelihood (I am using 1.96) instead of 2 (lines 295-298) This is the case when there is no information on symptoms so that there is 1 unknown parameter. When there is information on symptoms, there are two unknown parameters so the distance from the peak of the likelihood is twice that of the univariate case. https://doi.org/10.1371/journal.pone.0281710.r002
31 Jan 2023 Decision Letter - Rehana Naz, Editor Operational analysis for COVID-19 testing: Determining the risk from asymptomatic infections PONE-D-22-20932R1 Dear Dr. Mangel, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Rehana Naz Academic Editor PLOS ONE Reviewer's Responses to Questions Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation. Reviewer #1: All comments have been addressed Reviewer #2: All comments have been addressed Reviewer #3: All comments have been addressed ******** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Partly Reviewer #3: Yes ****** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes Reviewer #3: Yes ****** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes Reviewer #3: Yes ****** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes Reviewer #3: Yes ****** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: (No Response) Reviewer #2: The authors have improved lots of points in the revised version of the manuscript, especially the first and the third critical point. The manuscript is more preferable in the revised form. Reviewer #3: All concerns have been adequately addressed and the updated manuscript may be accepted for publication. ****** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: No Reviewer #3: Yes: Adnan Ahmed Khan ******** https://doi.org/10.1371/journal.pone.0281710.r003
Formally Accepted
3 Feb 2023 Acceptance Letter - Rehana Naz, Editor PONE-D-22-20932R1 Operational analysis for COVID-19 testing: Determining the risk from asymptomatic infections Dear Dr. Mangel: I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department. If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org. If we can help with anything else, please email us at plosone@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Prof Rehana Naz Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0281710.r004

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .