Identifying healthcare needs with patient experience reviews using ChatGPT

Jiaxuan Li; Yunchu Yang; Rong Chen; Dashun Zheng; Patrick Cheong-Iao Pang; Chi Kin Lam; Dennis Wong; Yapeng Wang

doi:10.1371/journal.pone.0313442

Peer Review History

Original SubmissionAugust 12, 2024
12 Aug 2024 Author Response https://doi.org/10.1371/journal.pone.0313442.r001
4 Sep 2024 Decision Letter - Jinran Wu, Editor PONE-D-24-34064Identifying Healthcare Needs with Patient Experience Reviews Using ChatGPTPLOS ONE Dear Dr. Cheong-lao Pang, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Oct 19 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org . When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols . Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols . We look forward to receiving your revised manuscript. Kind regards, Jinran Wu, PhD Academic Editor PLOS ONE Journal requirements: 1. When submitting your revision, we need you to address these additional requirements. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. Thank you for stating in your Funding Statement: This research was supported by the Macao Science and Technology Development Fund (funding IDs: 0048/2021/APD; 0088/2023/ITP2). Please provide an amended statement that declares all the funding or sources of support (whether external or internal to your organization) received during this study, as detailed online in our guide for authors at http://journals.plos.org/plosone/s/submit-now. Please also include the statement “There was no additional external funding received for this study.” in your updated Funding Statement. Please include your amended Funding Statement within your cover letter. We will change the online submission form on your behalf. 3. When completing the data availability statement of the submission form, you indicated that you will make your data available on acceptance. We strongly recommend all authors decide on a data sharing plan before acceptance, as the process can be lengthy and hold up publication timelines. Please note that, though access restrictions are acceptable now, your entire data will need to be made freely accessible if your manuscript is accepted for publication. This policy applies to all data except where public deposition would breach compliance with the protocol approved by your research ethics board. If you are unable to adhere to our open data policy, please kindly revise your statement to explain your reasoning and we will seek the editor's input on an exemption. Please be assured that, once you have provided your new statement, the assessment of your exemption will not hold up the peer review process. 4. We note that you have referenced "Mahapatra P." which has currently not yet been accepted for publication. Please remove this from your References and amend this to state in the body of your manuscript: (Mahapatra P. [Unpublished]”) as detailed online in our guide for authors http://journals.plos.org/plosone/s/submission-guidelines#loc-reference-style [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Partly Reviewer #2: Yes Reviewer #3: Yes Reviewer #4: Yes ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes Reviewer #3: Yes Reviewer #4: Yes ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes Reviewer #3: Yes Reviewer #4: No ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes Reviewer #3: Yes Reviewer #4: Yes ****** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: This work used the reviews to create Aspect-Based Sentiment Analysis (ABSA) templates, categorizing patient reviews into three categories that reflect patients' areas of concern. By introducing thought chains, the ABSA templates were embedded into prompts for ChatGPT, which was then used to identify patient needs. The results show that the proposed method achieved a weighted total precision of 0.907, which is outstanding compared to the direct narrative tasks in ChatGPT. Overall, this work is complete, but there is still room for improvement. My suggestions are as follows: 1.Lines 53-54 “but these techniques still struggle to pinpoint specific areas of patient dissatisfaction within individual reviews” : is there any literature to support this statement? 2.Lines 51 - 59：In this paragraph, the first part ("Recent research... individual reviews") discusses the limitations of deep learning and data mining approaches in analyzing patient reviews, but the second part ("Finally, some studies... patient experience") covers other topics. These two parts lack a clear logical connection. It is recommended to revise this paragraph to ensure clearer and more coherent logic. 3.Lines 112 -123 : The hyperparameters within ChatGPT are set to their default values. Is it possible to ensure that the parameters used are optimal without tuning them? Could the model's performance be improved by adjusting these hyperparameters? 4.The test size is 2000, out of a total data size of 504,198. This is a relatively small proportion (approximately 0.4%) for the test set, which might not provide a robust estimate of the model's performance on unseen data. 5.Lines 245- 246: For the test set (2000 samples), labeling was done manually. For larger datasets, relying on manual labeling is very inefficient. Are there any better methods to improve efficiency? 6.It is recommended to add an explanation of how the three metrics (Precision, Recall, and F1-weight) are used to evaluate the model's performance. For example, the higher the Precision, the better the model's performance. 7.Does the F1 score in Table 3 represent the weighted F1 score ? And is the F1 score in the Model Performance section also referring to the weighted F1 score? The authors should provide a clear description. Reviewer #2: The authors need more literatures to support their claim as in the paper there is a need to justify evidences with literatures that can prove most of the claim are true and can ensure that this is a worth area to research on. Reviewer #3: here are several review comments for the manuscript: 1 The manuscript is well-organized, and the objectives are clearly stated. However, the authors could enhance the clarity by providing a more detailed introduction to the significance of the study and its contribution to the existing literature. 2 the authors should elaborate on the methodology, particularly the process of creating ABSA templates and the rationale behind the chosen categories. Additionally, a discussion on the potential limitations of ChatGPT in understanding complex sentiment expressions would be beneficial. 3 Comparison with Existing Studies: The authors should provide a more comprehensive comparison with other studies that have used similar approaches or have addressed the same research questions. This will help position their work within the broader context of the field. 4 It is recommended that a professional editor review the manuscript for language improvements. Reviewer #4: This study explores the use of ChatGPT combined with Aspect-Based Sentiment Analysis (ABSA) templates to identify healthcare needs from patient experience reviews. The researchers analyzed 504,198 reviews from an online medical platform, categorizing patient feedback into three areas: patient experience, physician skills and efficiency, and infrastructure and administration. By embedding ABSA templates into ChatGPT prompts, the study achieved higher precision, recall, and F1 scores compared to direct narrative tasks in ChatGPT. The results suggest that this approach can effectively capture patient sentiments and provide valuable insights for healthcare resource allocation and policy evaluation. The study concludes that integrating LLMs with structured analysis techniques like ABSA can significantly enhance the understanding of patient needs in healthcare. I have attached a few comments, with the main concern being the use of an older version of ChatGPT, which has been documented to have lower performance both as a general LLM and specifically in healthcare-related studies. Given that the aim is to optimize performance while saving costs, and considering that time efficiency is also a factor in cost, I strongly recommend using the most advanced version available. Abstract: The first sentence of the objective section should be moved to the introduction. Introduction: 1 - In lines 55 and 58, please address the redundancy of the phrase "It has also been pointed out," which appears twice in close proximity. 2 - The last two paragraphs of the introduction currently summarize the paper and should be moved to the discussion section instead. Additionally, the introduction is quite lengthy. I suggest removing these parts and making the introduction more concise. 3 Methods – line 235: It's unfortunate that significant effort was invested while using ChatGPT-3.5, especially since numerous studies have already demonstrated that newer versions are significantly more accurate. Given the goal of "avoiding unnecessary waste of healthcare resources," as stated in the abstract, I would strongly recommend using the latest paid version, which is relatively affordable and would likely offer more cost-effective and precise results. Discussion: 4 - Innovative Aspect: The integration of ABSA templates with ChatGPT is well-executed, but the authors should more clearly emphasize what sets their approach apart from existing methods. approach innovative compared to existing methods. Highlighting the unique contributions more explicitly would make the manuscript's novelty clearer. ****** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean? ). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy . Reviewer #1: No Reviewer #2: Yes: Abdullahi Tunde Aborode Reviewer #3: No Reviewer #4: No ******** While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/ . PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org . Please note that Supporting Information files do not need this step. Attachments Attachment Submitted filename: comments.docx https://doi.org/10.1371/journal.pone.0313442.r002
Revision 1
17 Oct 2024 Author Response Dear Editor and Reviewers, Many thanks for your comments on our manuscript and providing us the opportunity of submitting a revised version for your consideration. We have addressed all the comments and made significant changes to the manuscript accordingly. The changes to the manuscript are red highlighted. Specific responses to each comment are detailed below. Reviewer 1#: General comments ============= This work used the reviews to create Aspect-Based Sentiment Analysis (ABSA) templates, categorizing patient reviews into three categories that reflect patients' areas of concern. By introducing thought chains, the ABSA templates were embedded into prompts for ChatGPT, which was then used to identify patient needs. The results show that the proposed method achieved a weighted total precision of 0.907, which is outstanding compared to the direct narrative tasks in ChatGPT. Overall, this work is complete, but there is still room for improvement. My suggestions are as follows: RESPONSE: We appreciate the reviewers' valuable suggestions and recognition of our work. Through which we have improved the quality of the paper, and hope that our revisions will address these issues. Specific comments ============= 1. Lines 53-54 “but these techniques still struggle to pinpoint specific areas of patient dissatisfaction within individual reviews”: is there any literature to support this statement? RESPONSE: We thank the reviewers for the suggestion that our previous lack of cited literature failed to support this sentence. In the latest version, we have cited two recent studies (references 43-44) from the last two years to support the sentence. 2. Lines 51 - 59: In this paragraph, the first part ("Recent research... individual reviews") discusses the limitations of deep learning and data mining approaches in analyzing patient reviews, but the second part ("Finally, some studies... patient experience") covers other topics. These two parts lack a clear logical connection. It is recommended to revise this paragraph to ensure clearer and more coherent logic. RESPONSE: Thanks to the reviewers' meaningful comments. We were missing a logical connection to the patient experience in this section. In the new version, we have added an introduction about the importance of patient experience and some connecting statements to make the context more logical. 3. Lines 112 -123 : The hyperparameters within ChatGPT are set to their default values. Is it possible to ensure that the parameters used are optimal without tuning them? Could the model's performance be improved by adjusting these hyperparameters? RESPONSE: Thanks to the reviewer for the comments. On the one hand, we wanted to see if LLM could accomplish this complex task without adjusting hyperparameters (the easiest to use version). On the other hand, the hyperparameters research on LLM is a hot and difficult research direction at present, especially the hyperparameters research on closed-source LLM. But we lacked a description of this in the paper, so we include this aspect in Limitations and discuss the possibility of a more detailed study in the future, including how to find the optimal hyperparameters for a specific task. 4. The test size is 2000, out of a total data size of 504,198. This is a relatively small proportion (approximately 0.4%) for the test set, which might not provide a robust estimate of the model's performance on unseen data. RESPONSE: Thanks to the reviewer's comments. We noticed this issue during our experiments, so we included Reliability and Stability Evaluation (Lines 282) for a secondary assessment of accuracy. The reason for the sparse test set is that the labeling process in the ABSA task is very difficult [1-3], and we already have one of the largest test set data in the same type of experiment (manual labeling). Therefore, in the latest version, we have included a description of this aspect in the Limitation section, as well as a discussion of the fact that in the future we can do related research on automatic labeling of LLMs to address the difficulty of labeling in complex tasks. [1]Hua Y C, Denny P, Wicker J, et al. A systematic review of aspect-based sentiment analysis: domains, methods, and trends. Artificial Intelligence Review, 2024, 57(11): 296. [2]Imani M, Noferesti S. Aspect extraction and classification for sentiment analysis in drug reviews. Journal of Intelligent Information Systems, 2022, 59(3): 613-633. [3]Chen Y, Liu S, Zhang X, et al. Automatically labeled data generation for large scale event extraction. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2017: 409-419. 5. Lines 245- 246: For the test set (2000 samples), labeling was done manually. For larger datasets, relying on manual labeling is very inefficient. Are there any better methods to improve efficiency? RESPONSE: Thanks to the reviewers for their suggestions. And as mentioned in the previous response, improving labeling efficiency is a pressing issue. Therefore, we discuss in Limitation about how to use LLM automatically labeled datasets in the future to improve labeling efficiency. We hope this will address your query. 6. It is recommended to add an explanation of how the three metrics (Precision, Recall, and F1-weight) are used to evaluate the model's performance. For example, the higher the Precision, the better the model's performance. RESPONSE: Thanks to the reviewer's suggestion. We omitted this explanation before. In the new version, we have included an explanation of how the three metrics specific to this task relate to model performance in line 293. 7. Does the F1 score in Table 3 represent the weighted F1 score ? And is the F1 score in the Model Performance section also referring to the weighted F1 score? The authors should provide a clear description. RESPONSE: Thanks to the reviewer for such a detailed comment. We have used a weighted F1 score in order to avoid sample imbalance, and have abbreviated it to an F1 score for the sake of the aesthetics of the table as well as the readability of the paper. But we forgot to introduce it, in the latest version, we have added this note and informed that the F1 score in the entire Result section is a weighted F1 score. Reviewer 2#: General comments ============= The authors need more literatures to support their claim as in the paper there is a need to justify evidences with literatures that can prove most of the claim are true and can ensure that this is a worth area to research on. RESPONSE: We thank the reviewers for their valuable suggestions. And we have made a complete revision to address this issue. In this latest version, we have included new literature in several sections of the text to support our conclusions and elaborations. The total number of new papers added is more than ten, and we hope that the reviewers' questions can be solved. Reviewer 3#: General comments Here are several review comments for the manuscript: RESPONSE: We thank the reviewers for their valuable comments. This comments greatly improved the quality of our paper. Specific comments 1. The manuscript is well-organized, and the objectives are clearly stated. However, the authors could enhance the clarity by providing a more detailed introduction to the significance of the study and its contribution to the existing literature. RESPONSE: Thanks to the reviewers' valuable comments. We have presented the significance of the study in more detail in the latest version, which mainly includes the solution to the difficulties of ABSA in training models and the solution ideas in complex tasks, etc. On the other hand, we also elaborates our value and contribution to the existing literature. 2. The authors should elaborate on the methodology, particularly the process of creating ABSA templates and the rationale behind the chosen categories. Additionally, a discussion on the potential limitations of ChatGPT in understanding complex sentiment expressions would be beneficial. RESPONSE: Thanks to the reviewers' valuable comments. We have elaborated the process of creating ABSA in more detail in our latest version and explained the rationale behind the selected categories with specific examples and the particularities of the Chinese healthcare environment. Finally, we have included in the Model Accuracy and Feasibility section of the Discussion section we describe the limitations of LLM in handling complex tasks and illustrate them using concrete examples. 3.Comparison with Existing Studies: The authors should provide a more comprehensive comparison with other studies that have used similar approaches or have addressed the same research questions. This will help position their work within the broader context of the field. RESPONSE: Thanks to the reviewers for their meaningful comments. We have included some new studies for comparison in our latest version to carry out a more in-depth discussion on the significance and contribution of our approach. Specifically, we have cited one study that addresses the same research question as ours and described their research method shortcomings. On the other hand, we cite a review article published in 2024 on the ABSA methodology, and in this way we compare all the studies using similar methodologies and describe the research gaps in the ABSA field on LLM. 4.It is recommended that a professional editor review the manuscript for language improvements. RESPONSE: Thanks to the reviewer's suggestion. We have addressed the language issues after fully reviewing our manuscript. Reviewer 4#: General comments This study explores the use of ChatGPT combined with Aspect-Based Sentiment Analysis (ABSA) templates to identify healthcare needs from patient experience reviews. The researchers analyzed 504,198 reviews from an online medical platform, categorizing patient feedback into three areas: patient experience, physician skills and efficiency, and infrastructure and administration. By embedding ABSA templates into ChatGPT prompts, the study achieved higher precision, recall, and F1 scores compared to direct narrative tasks in ChatGPT. The results suggest that this approach can effectively capture patient sentiments and provide valuable insights for healthcare resource allocation and policy evaluation. The study concludes that integrating LLMs with structured analysis techniques like ABSA can significantly enhance the understanding of patient needs in healthcare. I have attached a few comments, with the main concern being the use of an older version of ChatGPT, which has been documented to have lower performance both as a general LLM and specifically in healthcare-related studies. Given that the aim is to optimize performance while saving costs, and considering that time efficiency is also a factor in cost, I strongly recommend using the most advanced version available. RESPONSE: We thank the reviewers for such detailed and sincere comments. We re-experimented with the latest ChatGPT-4o-lastest at the first opportunity after receipt of the comments and reviewed the accuracy using the same multiplicity of methods. We changed all the descriptions of the older versions of ChatGPT in the article as well as changed the previous F1 score, accuracy and so on. Specific comments 1. Abstract: The first sentence of the objective section should be moved to the introduction. RESPONSE: Thanks to the reviewer's comments. We have moved this sentence to the Introduction section as you suggested and added two new citations to improve the reliability of this sentence. 2.Introduction: In lines 55 and 58, please address the redundancy of the phrase "It has also been pointed out," which appears twice in close proximity. RESPONSE: Thanks to the reviewers' valuable suggestions. We have addressed this writing error in the latest version, and we have checked the full text to ensure that similar errors do not occur again. 3. Introduction: The last two paragraphs of the introduction currently summarize the paper and should be moved to the discussion section instead. Additionally, the introduction is quite lengthy. I suggest removing these parts and making the introduction more concise. RESPONSE: Thanks to the reviewer's meaningful comments. Our introductory section was indeed too long resulting in a loss of readability, so we removed the paragraph to summarize the paper as per the reviewer's suggestion as a way to ensure brevity in the introductory section. 4. Methods: line 235: It's unfortunate that significant effort was invested while using ChatGPT-3.5, especially since numerous studies have already demonstrated that newer versions are significantly more accurate. Given the goal of "avoiding unnecessary waste of healthcare resources," as stated in the abstract, I would strongly recommend using the latest paid version, which is relatively affordable and would likely offer more cost-effective and precise results. RESPONSE: Thank you for such a detailed suggestion, and we agreed made a lot of sense. We redid the entire experiment with ChatGPT-4o. As you said, benefiting from the advantages of the new model in terms of cost and efficiency, we successfully completed the experiment in a short time and covered it with the original model in the full text. 5. Discussion: Innovative Aspect: The integration of ABSA templates with ChatGPT is well-executed, but the authors should more clearly emphasize what sets their approach apart from existing methods. approach innovative compared to existing methods. Highlighting the unique contributions more explicitly would make the manuscript's novelty clearer. RESPONSE: Thanks to the reviewers' valuable comments. In this latest version we cite a review article published in 2024 on ABSA methods, comparing the differences between all of them and describing issues such as the use of LLM in ABSA being a research gap. On the other hand, we cite the ABSA study, also on patient reviews, and describe how the challenge of this similar task lies in model training, among other things. Finally, we present the unique contributions and implications of our study through these new citations. https://doi.org/10.1371/journal.pone.0313442.r003
24 Oct 2024 Decision Letter - Jinran Wu, Editor Identifying Healthcare Needs with Patient Experience Reviews Using ChatGPT PONE-D-24-34064R1 Dear Dr. Pang, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Jinran Wu, PhD Academic Editor PLOS ONE Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation. Reviewer #1: All comments have been addressed Reviewer #2: All comments have been addressed ******** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ****** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: No ****** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ******** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: (No Response) Reviewer #2: The authors have addressed the comments as suggested with detailed feedback which have make the paper worthy. https://doi.org/10.1371/journal.pone.0313442.r004
Formally Accepted
Acceptance Letter - Jinran Wu, Editor PONE-D-24-34064R1 PLOS ONE Dear Dr. Pang, I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now being handed over to our production team. At this stage, our production department will prepare your paper for publication. This includes ensuring the following: * All references, tables, and figures are properly cited * All relevant supporting information is included in the manuscript submission, * There are no issues that prevent the paper from being properly typeset If revisions are needed, the production department will contact you directly to resolve them. If no revisions are needed, you will receive an email when the publication date has been set. At this time, we do not offer pre-publication proofs to authors during production of the accepted work. Please keep in mind that we are working through a large volume of accepted articles, so please give us a few weeks to review your paper and let you know the next and final steps. Lastly, if your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. If we can help with anything else, please email us at customercare@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. Jinran Wu Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0313442.r005

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .