PONE-D-22-12527SARS-CoV-2 reliably detected in frozen saliva samples stored up to one yearPLOS ONE

Dear Dr. Frediani,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

==============================

ACADEMIC EDITOR: As appended below, the reviewers have raised major concerns/critiques (reviewer # 3 is against publication) and suggested further justification/work to consolidate the findings. Do go through the comments and amend the MS accordingly.

==============================

Please submit your revised manuscript by Jul 04 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

A. M. Abd El-Aty

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf.

2. Please provide additional details regarding participant consent. In the ethics statement in the Methods and online submission information, please ensure that you have specified (1) whether consent was informed and (2) what type you obtained (for instance, written or verbal, and if verbal, how it was documented and witnessed). If your study included minors, state whether you obtained consent from parents or guardians. If the need for consent was waived by the ethics committee, please include this information.

If you are reporting a retrospective study of medical records or archived samples, please ensure that you have discussed whether all data were fully anonymized before you accessed them and/or whether the IRB or ethics committee waived the requirement for informed consent. If patients provided informed written consent to have data from their medical records used in research, please include this information.

3. Thank you for stating the following in Funding Section of your manuscript:

“This work was supported by the National Institute of Biomedical Imaging and Bioengineering RADx program, [Grant Number: 54 EB027690 02S1] WL,https://www.nibib.nih.gov/, and the National Center for Advancing Translational Sciences [Grant Number: UL1 TR002378](PI not an author),https://ncats.nih.gov/”

We note that you have provided additional information within the Acknowledgements Section that is not currently declared in your Funding Statement. Please note that funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form.

Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows:

” This work was supported by the National Institute of Biomedical Imaging and Bioengineering RADx program, [Grant Number: 54 EB027690 02S1] WL,https://www.nibib.nih.gov/, and the National Center for Advancing Translational Sciences [Grant Number: UL1 TR002378](PI not an author),https://ncats.nih.gov/

Funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript”

Please include your amended statements within your cover letter; we will change the online submission form on your behalf.

4. We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For more information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially sensitive information, data are owned by a third-party organization, etc.) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Partly

Reviewer #3: No

Reviewer #4: Partly

Reviewer #5: Partly

Reviewer #6: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: No

Reviewer #2: I Don't Know

Reviewer #3: N/A

Reviewer #4: No

Reviewer #5: No

Reviewer #6: No

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

Reviewer #5: Yes

Reviewer #6: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: No

Reviewer #3: Yes

Reviewer #4: No

Reviewer #5: Yes

Reviewer #6: No

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This is an interesting paper which presents some criticisms that should be fixed before its eventual publication

Major criticisms

No data are clearly presented regarding the analytical coefficient of variation of the assay used. Therefore, the eventual decrease after storage cannot be completely evaluated. This issue should be clarified

The figure should be improved as some variations should be better specified according to the previous criticism

Introduction:

the initial sentence seems inappropriate. The assay used is not a POCT and therefore, the real need is to develop accurate laboratory tests, not only POCT

References to be added, if possible:

a) Basso D, Aita A, Padoan A, Cosma C, Navaglia F, Moz S, Contran N, Zambon CF, Maria Cattelan A, Plebani M. Salivary SARS-CoV-2 antigen rapid detection: A prospective cohort study. Clin Chim Acta. 2021 Jun;517:54-59. doi: 10.1016/j.cca.2021.02.014.

b) Basso D, Aita A, Navaglia F, Mason P, Moz S, Pinato A, Melloni B, Iannelli L, Padoan A, Cosma C, Moretto A, Scuttari A, Mapelli D, Rizzuto R, Plebani M. The University of Padua salivary-based SARS-CoV-2 surveillance program minimized viral transmission during the second and third pandemic wave. BMC Med. 2022 Feb 23;20(1):96. doi: 10.1186/s12916-022-02297-1.

Reviewer #2: Abstract, line 65 – I would update that viability has not been reported on, rather than not studied as it is quite possible that it has been studied on in a number of settings but not specifically reported on.

Line 75 – I feel this opening sentence is not well aligned with the overall theme/message of the paper. The investigations in this paper do not involve POC technologies and apply to diagnostic development broader than just POC. I suggest that this introduction be reframed to better introduce the work and message in the paper.

Line 105 – please cite the paper on unsupervised collection devices mentioned here (https://doi.org/10.1186/s12879-022-07285-7)

Line 112 – please cite the paper, EUA or protocols.io on SalivaDirect for reference to this method (or more the#5 citation to the first mention of SalivaDirect on line 112 rather than after the PCR instrument).

Line 164/165 – remove gap between “- 6.7%”

Being a dualplex qPCR, it would be interesting for the authors to also report results for RP over time and how this compares to SARS-CoV-2 detection.

The authors are missing perhaps the earliest work on stability of unsupplemented raw saliva and SARS-CoV-2 detection and are likely more relevant than those currently included in the discussion: doi: 10.3201/eid2704.204199

The citation recommended for line 105 also reports on stability of SARS-CoV-2 RNA when cycled through various temperatures, and demonstrating that cold chain transport is not required.

The figures would be more informative, if in addition to the averages depicted, if the results for each pair could also be depicted. This would allow the reader to more robustly analyses how the pairs performed.

The authors fail to reflect on some of the large changes in Ct values between some of the pairs. Were re-tests double checked? Could anything different have happened during that time (primers, MMX)? Could any samples have not been tested properly the first time? Are any discrepancies more consistent per month perhaps further indicating a slight difference in that first test month?

Line 2 of the supplementary table shows an initial result of 0 – that doesn’t seem to be accurate. It could be helpful to have table either by month or by initial Ct.

Reviewer #3: The authors evaluated the reliability of the detection of SARS-CoV-2 RNA and antigen in frozen saliva samples stored for up to one year. The stability of saliva for the detection of SARS-CoV-2 RNA has been intensively investigated by researchers; however, the methods of preparation, preservation, and examination differed among the laboratories and high-quality investigations are still desired. Unfortunately, the current studies only showed the correlation between the first and second evaluations in a small number of samples, mainly in asymptomatic individuals with a single PCR assay and antigen testing.

Useful information as scientific data for universalization

(1) Sufficient number of samples for evaluation

(2) Stratified data

1) High viral load samples, moderate viral load samples, and low viral load samples

2) Conditions of preservation (4, -20, -80)

3) Data of several molecular assays, including commonly available assays

4) Difference of variants

Overall, the current research is insufficient to use as scientific data. Also, the quality of manuscript is low for an original article.

Minor point

CT ---> Ct

Line 112: Yale’s SalivaDirect dual-plexed RT-qPCR protocol: please add the reference or link

Line 115: -80C -� -80ºC

Reviewer #4: Enough information is presented (either in the paper or in cited references) to enable other investigators to replicate the work.

The presentation of the results could be improved in several ways, most notably by placing the numerical comparisons in a table, rather than presenting them as text.

The authors have drawn some strange conclusions based on the Ct values for the assays. IN comparing mean Ct values over time, they have not presented confidence intervals for the means, and have made statements about changes which seem likely to me to be “noise.” Furthermore, they seem to suggest that lower Ct values represent lower reliability, which isn’t true at all. They state that “January, the second oldest samples, displayed low correlation and reliability.” While the figure supports this conclusion, the authors provide no explanation for this odd behavior (error in assay performance? Specimen mishandling?).

The Figure could be significantly improved. There is no justification for the lines connecting the point estimates. Each point estimate should be accompanied by confidence intervals.

The authors have described antigen concentration assessment, and presented some verbiage in the results section, without presenting any sort of formal analysis. I suppose the comments are supported by data in the supplementary Excel spreadsheet, but I would be much happier if there were to be a more formal presentation of what the data say.

Reviewer #5: The paper is concise and straightforward, the conclusions are valid. The only part that is missing is alignment with MIQE guidelines for quantitative real-time PCR (DOI: 10.1373/clinchem.2008.112797). Understandably, the authors used commercial kits. However, if the information on validation of the kits is available, it is worth including it into the manuscript.

Another comment, the mean values in Figure 1 are given without error bars and n values for the number of samples. From supplementary data it is hard to understand how many samples were available for retesting for each month.The significance of the difference in the January sample should be reported by p-value.

Reviewer #6: The manuscript submitted by Dr. Frediani and entitled “SARS-CoV-2 reliably detected in frozen saliva samples stored up to one year” evaluates the long-term stability of frozen saliva specimens for the detection of SARS-CoV-2. In this work 87 specimens that were previously positive for SARS-CoV-2 and frozen and -80C were retested. The results of the specimens were then compared to the initial run to look at specimen stability. In general they found stable specimens for PCR with average decreases after the freeze thaw. The results of the study are useful as saliva for respiratory diagnostics is becoming more common due to the pandemic and understanding the matrix’s long-term stability will be valuable for laboratories and industry to validate novel assays when the viral target is low. However, in its current state there are some clarifications needed and re-writing necessary for publication. Here are my suggestions:

Major Comments

Ln132: Why would you create a CT value when no value was obtained. These should just not be run on statistical analysis since giving them a point value. When observing your data in figure 1 and in the averages, the freeze thaw seemed to improve detection based on the lower CT. Would this be more pronounced if the negative specimens were removed. It is also important to discuss these missed samples and the CT from the initial test. Were they near the LoD.

From the data trend, it almost appears that a freeze thaw improves the sensitivity of the assay, which has been a discussion in the field and possible concern for the FDA in evaluating retrospecitive specimens. It would be beneficial if a small subset of new specimens could be frozen and tested after 24 hours of freeze thaw to determine if

Ln124 and throughout: I would suggest reviewing the manuscript for conversational and indirect language. As an example, ln 124 states” the original chosen 10 could not be re-tested for some reason”. I would remove “for some reason” and add a sentence of the numbers that were not tested and reasons. This continues into more of the methods, which make it a bit unclear of how the specimens were tested. For example, n the sample selection when 10 were taken from December and 10 from January are these tested at the same time so one batch is X-months frozen and the January batch is X-1 months old or were they all stored for 12 months prior to testing?

Minor comments

Ln104: What is meant by compliance (i.e. was this approval via the institutions IRB)?

I would consider changing the term re-test to thawed specimens or something similar. When I read re-test I am thinking of a possible repeat for a test that was invalid.

Link figures in text when indicated and presenting data.

As there is only 87 data points I think it would be interesting to see the N1 CT values as a dots where the samples are on X axis CT on Y and 2 points for each sample so we can see the spread of CT values for individual samples and not as an average.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

Reviewer #3: No

Reviewer #4: No

Reviewer #5: No

Reviewer #6: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PONE-D-22-12527R1SARS-CoV-2 reliably detected in frozen saliva samples stored up to one yearPLOS ONE

Dear Dr. Frediani,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

==============================

ACADEMIC EDITOR: IN addition to the comments raised by the diligent reviewers, please amend the following:1- Full vendor details should include, company, city (state), and country2- Tables should be in landscape not portrait layout3- Conclusion should be in a separate section4- Study strengths and limitations should be in a separate section, headed as addressed. It should be ahead of the conclusion.  

==============================

Please submit your revised manuscript by Sep 05 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

A. M. Abd El-Aty

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

Reviewer #3: All comments have been addressed

Reviewer #4: (No Response)

Reviewer #6: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: No

Reviewer #4: Partly

Reviewer #6: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

Reviewer #6: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

Reviewer #6: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: (No Response)

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

Reviewer #6: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The authors made a great job in answering raised criticisms

From my side the paper is suitable for a publication

Reviewer #2: I commend the authors for thoroughly addressing each of the reviewers comments. The paper is much stronger and adds valuable data to the literature.

Reviewer #3: "SARS-CoV-2 reliably detected in frozen saliva samples stored up to one year"

No additional comments for the current manuscript.

Reviewer #4: The authors have adequately responded to all of my comments except one - what went wrong with the January specimens? Understanding this seems very important to supporting the conclusions of the paper. Things look good for 8 of 9 months, but this is not a level that I generally would be willing to accept in the clinical laboratory, nor would I expect a regulatory agency to accept as adequate in any part of the developed world. An explanation, or at least a speculation on what went wrong with these samples seems very important to me.

Reviewer #6: Ln 90 what do you mean by simple random samples? I would suggest removing the word simple.

All other comments have been addresed.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Mario Plebani

Reviewer #2: No

Reviewer #3: No

Reviewer #4: No

Reviewer #6: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

SARS-CoV-2 reliably detected in frozen saliva samples stored up to one year

PONE-D-22-12527R2

Dear Dr. Frediani,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

A. M. Abd El-Aty

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #4: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #4: (No Response)

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #4: (No Response)

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #4: (No Response)

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #4: (No Response)

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #4: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #4: No

**********