PONE-D-24-03994Self-Adaptive Teacher-Student Framework for Colon Polyp Segmentation from Unannotated Private Data with Public Annotated DatasetsPLOS ONE

Dear Dr. Dai,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

==============================

ACADEMIC EDITOR:  Does the information on segmentation in areas other than the colon polyp not enter into this consideration?

==============================

Please submit your revised manuscript by May 18 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Kazunori Nagasaka

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at 

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and 

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please note that PLOS ONE has specific guidelines on code sharing for submissions in which author-generated code underpins the findings in the manuscript. In these cases, all author-generated code must be made available without restrictions upon publication of the work. Please review our guidelines at https://journals.plos.org/plosone/s/materials-and-software-sharing#loc-sharing-code and ensure that your code is shared in a way that follows best practice and facilitates reproducibility and reuse.

3. Thank you for stating the following financial disclosure: 

"This work is supported by the research funding of Anhui Medical University (2022xkj105)."

Please state what role the funders took in the study.  If the funders had no role, please state: ""The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript."" 

If this statement is not correct you must amend it as needed. 

Please include this amended Role of Funder statement in your cover letter; we will change the online submission form on your behalf.

4. Thank you for stating the following in the Acknowledgments Section of your manuscript: 

"This work is supported by the research funding of Anhui Medical University (2022xkj105)."

We note that you have provided funding information that is not currently declared in your Funding Statement. However, funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form. 

Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows: 

"This work is supported by the research funding of Anhui Medical University (2022xkj105)."

Please include your amended statements within your cover letter; we will change the online submission form on your behalf.

5. We note that you have indicated that there are restrictions to data sharing for this study. For studies involving human research participant data or other sensitive data, we encourage authors to share de-identified or anonymized data. However, when data cannot be publicly shared for ethical reasons, we allow authors to make their data sets available upon request. For information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. 

Before we proceed with your manuscript, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially identifying or sensitive patient information, data are owned by a third-party organization, etc.) and who has imposed them (e.g., a Research Ethics Committee or Institutional Review Board, etc.). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. Please see http://www.bmj.com/content/340/bmj.c181.long for guidelines on how to de-identify and prepare clinical data for publication. For a list of recommended repositories, please see https://journals.plos.org/plosone/s/recommended-repositories. You also have the option of uploading the data as Supporting Information files, but we would recommend depositing data directly to a data repository if possible.

Please update your Data Availability statement in the submission form accordingly.

6. Please ensure that you refer to Figure 10 in your text as, if accepted, production will need this reference to link the reader to the figure.

Additional Editor Comments:

Dear Authors,

Thank you very much for submitting your research to PLOS ONE.

Ou decision is major revision.

Please reply to all reviewer's comments and revise the manuscript accordingly.

Sincerely,

Kazunori Nagasaka

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: No

Reviewer #3: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This paper proposed a self-adaptive teacher-student framework for polyp segmentation, to bridge the gap between the annotated datasets and the unannotated datasets. The paper is well-structured but still need further improvement.

1. The major problem of this paper is the experiment on the private dataset. Among the public datasets, CVC-ClinicDB has 612 cases, CVC-ColonDB has 380 cases and Kvasir has 998 cases. However, the private dataset only has 15 cases and only 3 cases were used for model testing. Since this paper aims to solve the data distribution gap between the well-annotated public datasets and the unannotated private dataset. I am afraid such a small-scale cannot cover the heterogeneity of the polyps. How to select the sample to avoid sample selection bias? Authors should carefully response this comment, otherwise I may not be able to accept this paper due to the unconvincing experimental results.

2. In Fig. 6, the results from different methods seems have no different. In case 2, I also wonder whether authors show the same frame? Because the image content of the highlighted areas in different approaches are obvious different. But the segmentation results seem all the same. I guess the red boundaries might be the GT, then where are the segmentation results? Can authors explain this?

2. Why two proposed modules UDFusion and GANet are not introduced in abstract?

3. The captions of figures should deliver more information to help readers better understand the technical novelties, especially Fig. 2 and Fig. 3

4. Besides SAM, authors only compared their proposed model with four competitors proposed in 2022. I suggest comparing more recent approaches proposed in 2023.

5. English should be improved.

For example in abstract: “builds pseudo labels” should be “generates pseudo labels”

“innovatively bridging the gap” “innovatively” should be removed.

“This strategy mainly dynamically” “mainly” should be removed.

Reviewer #2: This paper introduces the Self-Adaptive Teacher-Student (SATS) framework, an unsupervised approach for colon polyp segmentation based on a teacher-student architecture. The SATS method comprises two primary phases. In the initial phase, pseudo-labels are generated by training three teacher networks on three existing public datasets. Using unannotated private data as input, the Uncertainty and Distance Fusion (UDFusion) method processes the outputs of teacher networks to obtain adaptive pseudo-labels. The second phase involves training a student network. Pseudo-labels generated in the prior phase are utilized to train a Grainy Attention Network (GANet), ensuring accurate identification and segmentation of colon polyps. Despite achieving satisfactory results, the paper still faces challenges in terms of method description and experimental aspects:

Comment #1:

Is the term "Teacher-Student" framework appropriate for this paper? Because this study involves a two-step unsupervised semantic segmentation process. Initially, three networks are trained using three public datasets, and pseudo-labels are generated using these trained networks. Subsequently, a separate network is trained using these pseudo-labels. The so-called teacher-student networks do not participate in the same framework for training and prediction. Moreover, they share identical structures.

Comment #2:

It appears that Figure 1(b) lacks a specific explanation of the feature distribution plot. Could you please clarify the meanings of the horizontal and vertical axes?

Comment #3:

Is the notation 'wivi(p, q)' on line 158 written incorrectly and should it be 'wi(p, q)'?

On line 173, is the description 'of at (p, q)' correct?

Comment #4:

The Uncertainty and Distance Fusion (UDFusion) method calculates weights on a per-pixel basis, lacking consideration for the correlation between pixels. Does this lead to the pseudo-label images losing contextual information of spatial pixels? It is hoped that the authors can provide an explanation for this issue.

Comment #5:

Formulas 6 and 7 provide limited descriptions of the uncertainty weight. Can there be an appropriate addition to the explanation of the interpretability of the uncertainty weight and the reasons behind its design?

Comment #6:

The correspondence between the modules in the architecture of Figure 3 and the described content in the text is not clearly illustrated. Additionally, the explanation in the text regarding the 'foreground attention' and 'background attention' of the Refine Transformer Block, and how they continuously correct mispredicted areas, is not sufficiently thorough.

Comment #7:

In Formula 11, why is 'zd' subtracted from 'zfg' and added to 'zbg'? Both 'zfg' and 'zbg' are calculated from the same two feature maps, with the only difference being the sign choice for Pd+1. What is the significance of the dual branches?

Comment #8:

In Figure 3, there is an input 'Od+1' in the 'zfg' branch, but there is no explanation for the 'Od+1' input from Figure 3 in Formula 11.

Comment #9:

Most of the visualization examples in Figure 4 are cases where the polyp shapes have smooth and regular boundaries. How does the SATS method perform in cases with complex boundaries? In Figure 4, for cases 1 and 3, the segmentation edges obtained by the method in this paper are not clear and appear mosaic-like. What could be the reason for this?

Comment #10:

In Table 1, were the other comparison experiments conducted using the same networks as those proposed in this paper, and were they also trained using the three public datasets? How was fairness ensured in the comparison? How do the parameter count and complexity of the compared methods compare? It is recommended to include some networks with similar attention structures in the comparison experiments for more convincing results.

For example:

1)Yu C, Xiao B, Gao C, et al. Lite-hrnet: A lightweight high-resolution network[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021: 10440-10450.

2)Haiqiang Liu, Meibao Yao*, Xueming Xiao, Bo Zheng, Hutao Cui. MarsScapes and UDAFormer: a Panorama Dataset and a Transformer-based Unsupervised Domain Adaptation Framework for Martian Terrain Segmentation[J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, vol. 62, pp. 1-17, Art no. 4600117, https://doi.org/10.1109/TGRS.2023.3343109.

Comment #11:

The Ablation studies lack sufficient evidence and justification for the roles and necessity of the Identity Transformer and Refine Transformer modules.

Reviewer #3: **Summary**

The authors propose a student-teacher framework for semi-supervised segmentation of colon polyps on unlabeled video data by leveraging dynamic pseudo-label generation across teacher networks trained on multiple datasets. To measure distribution discrepancy between public labeled and private unlabeled datasets, reconstruction similarity is used a metric from public data trained auto-encoders. To enhance segmentation prediction from pseudo labels, the authors introduce refinement modules to suppress false predictions and incorporate multi-scale information to retain anatomical details. Evaluation using three public datasets for pseudo-label generation by training on a single private unlabeled dataset reveal significant gains over prior art, validating the intuition behind the designs.

**Strengths**

- The authors address a relevant task in colon polyp segmentation on unlabeled data. It is very intuitive to leverage existing labeled data to generate pseudo-labels for unlabeled samples.

- An simple and effective approach is introduced to measure distribution shifts between datasets based on generative modeling of data. Given the estimated shifts, the author leverage it as prior to enhance the learning of a student network.

- Experiments are quite extensive i.e., leveraging three public datasets provides evidence to support the claims.

**Weaknesses**

- While I appreciate the experimental setting, It is unclear if the compared methods were trained on the full public dataset (i.e., CVC-ClinicDB + CVC-ColonDB + Kvasir) or a select dataset. Since the proposed approach uses all three, it is important to explicitly highlight this point in the paper.

- As the proposed approach leverages multiple teachers, it is important to establish a lower-bound for the sensitivity of the approach by using fewer (≤ K) teachers/datasets. This can better contextualize the source of improvement since it is not clear if the approach will work with a single teacher. Moreover, the evaluation does not include popular semi-supervised approaches. (see references below).

- Yang et al. Revisiting Weak-to-Strong Consistency in Semi-Supervised Semantic Segmentation. CVPR 23

- Zhao et al. Augmentation Matters: A Simple-yet-Effective Approach to Semi-supervised

Semantic Segmentation. CVPR 23

- Chen et al. C-CAM: Causal CAM for Weakly Supervised Semantic Segmentation on Medical Image. CVPR 22

- The motivation behind using the L_tversky loss is not elaborated clearly. Additionally, the is no citation/reference to the original. More details on this objective need to be clarified.

- The writing and descriptions are not verbose. For instance, several instances of lose wording and unclear descriptions e.g., L380-381 in the selection experiment section. L400 should be ‘Evaluation metrics’.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Meibao Yao

Reviewer #3: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PONE-D-24-03994R1Self-Adaptive Teacher-Student Framework for Colon Polyp Segmentation from Unannotated Private Data with Public Annotated DatasetsPLOS ONE

Dear Dr. Dai,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Aug 01 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Kazunori Nagasaka

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments:

Dear Authors,

Thank you for your submission.

As you can see in the comments, you still need some modifications regarding the manuscript.

Please consider it and we look foward to your revised manuscript.

Sincerely,

Kazunori Nagasaka

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Authors have addressed all the comments. I have no further comment. This paper is ready to be published.

Reviewer #2: Thank you to the authors for providing detailed responses and revisions to the previous concerns. Most of the issues in the paper have been addressed. However, there are still a few questions that need further clarification:

Comment #1:

In the experimental results section, it seems that the distribution bias among the public datasets leads to unstable prediction results. Did the authors decide the selection of the three public datasets with any specific criteria in mind, or were any pre-processing methods used to enhance the stability of pseudo-label generation?

Comment #2:

In line 176, it is mentioned that the UDFusion module uses ui to assess the reliability of pseudo-labels through contextual information. How is this contextual information represented? Is there a direct connection between formula (1) and the unstable distribution in the final visualized results?

Comment #3:

In line 189, there are two colons “:”. Is this a typographical error?

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Self-Adaptive Teacher-Student Framework for Colon Polyp Segmentation from Unannotated Private Data with Public Annotated Datasets

PONE-D-24-03994R2

Dear Dr. Dai,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Kazunori Nagasaka

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Dear Authors,

Thank you so much for your submission to Plos One.

I am pleased to inform you that we made a decision as "Accept" for your manuscript.

Your study is very useful in the field and we look forward to your future work.

Sincerely,

Kazunori Nagasaka

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #2: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: No

**********