A consensus-based decision model for assessing the health systems

Yang Xu; Kin Keung Lai; W. K. J. Leung

doi:10.1371/journal.pone.0237892

Peer Review History

Original SubmissionJuly 7, 2020
24 Jul 2020 Decision Letter - Dragan Pamucar, Editor PONE-D-20-20783 A Consensus-based Decision Model for Assessing the Health Systems PLOS ONE Dear Dr. Lai, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Sep 07 2020 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols We look forward to receiving your revised manuscript. Kind regards, Dragan Pamucar Academic Editor PLOS ONE Journal Requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. Thank you very much for your submission to PLOS ONE. Before we proceed, we kindly ask that you address the following: Please explain the rationale for the development of your method in light of recent research in this area, clearly indicating which problem with existing method you are addressing. Please clearly report at the beginning of your methods or results section which the key performance measures were to establish validity and utility of your method. Please also report clearly which statistical analysis was used to establish robustness of performance measures. Please note that PLOS ONE requires that experiments, statistics, and other analyses must be performed to a high technical standard and described in sufficient detail to allow for reproducibility of the study (http://journals.plos.org/plosone/s/criteria-for-publication#loc-3). To demonstrate the performance of the method, we would expect comparisons to be drawn between existing state-of-the-art methods. Thank you for your attention to these queries. 3. Thank you for including your funding statement; "The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript." At this time, please address the following queries: Please clarify the sources of funding (financial or material support) for your study. List the grants or organizations that supported your study, including funding received from your institution. State what role the funders took in the study. If the funders had no role in your study, please state: “The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.” If any authors received a salary from any of your funders, please state which authors and which funders. If you did not receive any funding for this study, please state: “The authors received no specific funding for this work.” Please include your amended statements within your cover letter; we will change the online submission form on your behalf. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author* 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ****** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: Dear editor, Thank you for sending me for review the paper “A Consensus-based Decision Model for Assessing the Health Systems”. This paper intends to investigate Entropy method for determining criteria weights and TOPSIS evaluation method group decision making. The authors implemented hybrid MCDM framework for evaluation in healthcare field. Presented methodology has a great potential in decision making process and I am giving a support to the authors for investigation this topic. The strengths of this paper are: Relevant topic; Flow of the paper; and Explanation of the methods. However, the author(s) need to consider the following points as limitation or further scope for refining the paper: - Introduction should be clearly stated research questions and targets first. Then answer several questions: Why is the topic important (or why do you study on it)? What are the research questions? What are your contributions? Why is to propose this particular methods? The last two questions are answered in some parts in the Introduction section. But, the answer is not presented in a proper way. You should provide more information in this regard. - Need to highlight the novelty of study in the introduction. - I suggest authors to clearly summarize what specific advantages brings your approach. Enrich your Introduction section with more explanation: Why do you present this approach? Why you use Entropy method for criteria weighting and not the other objective methods like CRITIC or FANMA method? - Why did you use objective methodology for determining criteria weights and not subjective methodologies like AHP, BWM, LBWA etc? I must stress that there are numerous limitations of Entropy method that authors should have on their minds. For example, if only one value in decision matrix is above/below the other values for 20-30% (within the same criteria) leads drastically to increasing criteria weight of that criteria. That limitation is presented in almost all objective methods. In my opinion this facts should not be neglected. So, why we need objective weights? - Remove lumped references. All references cited in the text should be explained and discussed in the text. Remove some old references published before 2017-2018. Also, literature review should be presented in a better way. You should discuss application of various MCDM tools different fields, especially in healthcare field. You should update your literature review with a papers published in last two-three years, and remove old references. I suggest authors to read and cite below interesting references: Roy, J., Adhikary, K., Kar, S., & Pamucar, D. (2018). A rough strength relational DEMATEL model for analysing the key success factors of hospital service quality. Decision Making: Applications in Management and Engineering, 1(1), 121-142.; Badi, I., Abdulshahed, A., Shetwan, A., & Eltayeb, W. (2019). Evaluation of solid waste treatment methods in Libya by using the analytic hierarchy process. Decision Making: Applications in Management and Engineering, 2(2), 19-35. Biswas, S., Bandyopadhyay, G., Guha, B., & Bhattacharjee, M. (2019). An ensemble approach for portfolio selection in a multi-criteria decision making framework. Decision Making: Applications in Management and Engineering, 2(2), 138-158. - Add flowchart of proposed methodology and follow that flowchart steps in case study. - Case study should be better organized. The calculations should be deeply presented and follow the methodology presented in methodology section. Add more deep calculations in case study section. - Add sensitivity analysis and validation of the results. - The problem on which this present method is applied has significant social and managerial implications. How the method can address those implications need to be included. - Conclusion- Add future scope. Also, how the proposed method can be applicable to other real life problems need to be mentioned. Add limitations of proposed model. Do not use bullets or numerations in this section. I will review revised paper with my pleasure. Reviewer #2: The paper analyses an actual topic and it is interesting to potential readers. The paper is well prepared, it contains all required parts of a scientific paper. I have only one critical comment. References are up to 2016. Therefore the authors should update the literature review. ****** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review?** For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: No [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0237892.r001
Revision 1
31 Jul 2020 Author Response Many thanks for your valuable comments on our manuscript. We accordingly make the revisions and mark them using RED in the paper. Reviewer #1: Dear editor, Thank you for sending me for review the paper “A Consensus-based Decision Model for Assessing the Health Systems”. This paper intends to investigate Entropy method for determining criteria weights and TOPSIS evaluation method group decision making. The authors implemented hybrid MCDM framework for evaluation in healthcare field. Presented methodology has a great potential in decision making process and I am giving a support to the authors for investigation this topic. The strengths of this paper are: Relevant topic; Flow of the paper; and Explanation of the methods. However, the author(s) need to consider the following points as limitation or further scope for refining the paper: - Introduction should be clearly stated research questions and targets first. Then answer several questions: Why is the topic important (or why do you study on it)? What are the research questions? What are your contributions? Why is to propose this particular methods? The last two questions are answered in some parts in the Introduction section. But, the answer is not presented in a proper way. You should provide more information in this regard. Many thanks for your comments. (1) We discuss the importance of this study in paragraph 1. (2) The research questions are described in Page 5: (1) how to define the individual stakeholders of health systems? (2) how to achieve the consensus among different stakeholders? (3) the contributions and novelties are summarized in Pages 5&6. (4) The present paper is motivated by the observation that in the process of performance assessment, not only the preferences associate with evaluation criteria may exhibit a substantial degree of variability, but also different members of the decision committee have different opinions, which are extremely difficult to achieve a group consensus (Csaszar and Eggers, 2013; Melkonyan and Safra, 2016). In this sense, this work proposes a consensus-based decision model to assess the health systems. - Need to highlight the novelty of study in the introduction. Many thanks for your comments. The contributions and novelties of this study are summarized in Section 1. - I suggest authors to clearly summarize what specific advantages brings your approach. Enrich your Introduction section with more explanation: Why do you present this approach? Why you use Entropy method for criteria weighting and not the other objective methods like CRITIC or FANMA method? Many thanks for your comments. There exist many other objective weight elicitation approaches to assessing the health systems. We choose the three approaches for illustrating the working procedure of the proposed consensus-based decision model. The other objective methods would be investigated in future research. - Why did you use objective methodology for determining criteria weights and not subjective methodologies like AHP, BWM, LBWA etc? I must stress that there are numerous limitations of Entropy method that authors should have on their minds. For example, if only one value in decision matrix is above/below the other values for 20-30% (within the same criteria) leads drastically to increasing criteria weight of that criteria. That limitation is presented in almost all objective methods. In my opinion this facts should not be neglected. So, why we need objective weights? Many thanks for your comments. The main advantage of these objective approaches is the reduction of decision bias in terms of ignoring the subjective judgments of the individual stakeholders. Objective criteria weight determination approaches are usually applicable when individual stakeholders disagree on the exact values of criteria weights (Yu & Lai, 2011). Speciﬁcally, the rationale behind objective criteria weight determination approaches is that the importance degree of a criterion is a function of the information conveyed by this criterion, relative to a whole set of alternatives. - Remove lumped references. All references cited in the text should be explained and discussed in the text. Remove some old references published before 2017-2018. Also, literature review should be presented in a better way. You should discuss application of various MCDM tools different fields, especially in healthcare field. You should update your literature review with papers published in last two-three years, and remove old references. I suggest authors to read and cite below interesting references: Roy, J., Adhikary, K., Kar, S., & Pamucar, D. (2018). A rough strength relational DEMATEL model for analysing the key success factors of hospital service quality. Decision Making: Applications in Management and Engineering, 1(1), 121-142.; Badi, I., Abdulshahed, A., Shetwan, A., & Eltayeb, W. (2019). Evaluation of solid waste treatment methods in Libya by using the analytic hierarchy process. Decision Making: Applications in Management and Engineering, 2(2), 19-35. Biswas, S., Bandyopadhyay, G., Guha, B., & Bhattacharjee, M. (2019). An ensemble approach for portfolio selection in a multi-criteria decision making framework. Decision Making: Applications in Management and Engineering, 2(2), 138-158. Many thanks for your comments. The literature part has been updated according to your comments and some other new publications. - Add flowchart of proposed methodology and follow that flowchart steps in case study. Many thanks for your comments. The flowchart is added to Section 2. - Case study should be better organized. The calculations should be deeply presented and follow the methodology presented in methodology section. Add more deep calculations in case study section. Many thanks for your comments. We further elaborate the computation process of the proposed methodology in case study section (Section 3). - Add sensitivity analysis and validation of the results. Many thanks for your comments. For the purpose of validating the proposed consensus-based decision model to assess health systems, the Spearman’s rank correlation coefficient is computed and discussed. Please refer to Section 3. - The problem on which this present method is applied has significant social and managerial implications. How the method can address those implications need to be included. Many thanks for your comments. The implications are added in the last paragraph of Section 3. - Conclusion- Add future scope. Also, how the proposed method can be applicable to other real life problems need to be mentioned. Add limitations of proposed model. Do not use bullets or numerations in this section. Many thanks for your comments. Limitations and future scope are added to Section 4. Reviewer #2: The paper analyses an actual topic and it is interesting to potential readers. The paper is well prepared, it contains all required parts of a scientific paper. I have only one critical comment. References are up to 2016. Therefore the authors should update the literature review. Many thanks for your valuable comments. We have added several references after 2016 in Section 1. Attachments Attachment Submitted filename: Response to reviewer 2.docx https://doi.org/10.1371/journal.pone.0237892.r002
5 Aug 2020 Decision Letter - Dragan Pamucar, Editor A Consensus-based Decision Model for Assessing the Health Systems PONE-D-20-20783R1 Dear Dr. Lai, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Dragan Pamucar Academic Editor PLOS ONE Additional Editor Comments (optional): Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation. Reviewer #1: All comments have been addressed Reviewer #2: All comments have been addressed ******** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ****** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes ****** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ****** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: All the reviewers' comments have been addressed carefully and sufficiently, the revisions are rational from my point of view, I think the current version of the paper can be accepted. Reviewer #2: Several new references are added, but there could be more of them. Nvertheless, the paper can be accepted. ****** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review?** For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: No https://doi.org/10.1371/journal.pone.0237892.r003
Formally Accepted
7 Aug 2020 Acceptance Letter - Dragan Pamucar, Editor PONE-D-20-20783R1 A Consensus-based Decision Model for Assessing the Health Systems Dear Dr. Lai: I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department. If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org. If we can help with anything else, please email us at plosone@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. Dragan Pamucar Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0237892.r004

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .