Systematic review on cost-effectiveness analysis of school-based oral health promotion program

Thinni Nurul Rochmah; Aulia Ramadhani; Taufan Bramantoro; Lucindari Gea Permata; Tin Zar Tun

doi:10.1371/journal.pone.0284518

Peer Review History

Original SubmissionJanuary 11, 2023
15 Feb 2023 Decision Letter - Fahad Umer, Editor PONE-D-23-00755Cost-Effectiveness Analysis of school-based oral health promotive and preventive program for elementary school children: A systematic reviewPLOS ONE Dear Dr.Thinni Nurul Rochmah Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Apr 01 2023 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Fahad Umer Academic Editor PLOS ONE Journal Requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: N/A Reviewer #2: Yes ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: No Reviewer #2: No ****** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: This is a systematic review of cost-effectiveness studies of school-based oral health programs for elementary school children. It found 5 studies (consisting of 8 programs) out of which 4 programs were found to be cost saving and the other 4 potentially cost-effective. The study is registered in Prospero and generally follows the PRISMA-standard. However, I found some weaknesses with it and have some comments. The title is misleading, you do not do a CEA, you do a SR on other CEAs. In several parts of the study, the English language needs to be improved. The setting of the oral health program is not presented or discussed in detail, even though that is highly influential when performing promotive and preventive programs. School systems in different countries tend to differ. Negative cost-effectiveness ratios should not be presented in values as such ratios can’t be interpreted (more than saying that it is dominant). Similar, the use of terms such as “most cost-effective” is not correct, which you use in your research question. You state that you assess the risk of bias, but I can’t find any presentation of this. Your study quality assessment is more of an indicator whether certain aspects are covered in the analysis, rather than analysing the risk-of-bias of those aspects within the study. Some parts of the result sections belong to the discussion. I was surprised that only five studies were included. I think you need to present the reference to the 72 excluded full text studies in appendix and present what main criteria they failed to fulfil. In my opinion the use of the dominance ranking matrix analysis does not provide any additional valuable information in this systematic review. Hence, I think Table 4 and 5 can be omitted, this information is already presented in Table 3. There are unnecessary part in the discussion, you don’t have to explain what a CEA and an ICER is or how to interpret it, that should (if needed) have been presented in the introduction. However, you need to compare your findings with other studies. There are several systematic reviews in child oral health programs that should have been discussed here (even though they have not used school as a strict criterion for the programs), for example by Hettiarachchi et al 2018, Fraihat et al 2019, Eow et al 2019, Davidson et al 2021. You write that there are several limitations with the study, but you only mention two such ones (variety of health indicators and your searching keywords). I think there are other more important weaknesses such as lack of risk-of-bias-analysis, presentation of the settings, potential missing studies etc. Reviewer #2: Thank you for addressing a very important topic for the economic evaluation of different preventive and oral health promotion programs in elementary school-going children. The manuscript is adequately written however, there is some room for improvement. Attached below are the comments for your reference: Major points: The manuscript is quite wordy and should be significantly reduced in length. There are several formatting errors, especially the referencing. References should be cited at the end of the statement and after full stop. Try to write the manuscript in third-person past tense and get it reviewed by someone proficient in English writing. Introduction: Very well written; writing is supported by evidence. Line 48-51: There is redundancy, can be rephrased to a single statement. Line 64: Reference should never be in the middle of the statement. Some details about cost-effectiveness analysis and its utility can be added here in the introduction. Methods: Fairly adequate description of the methodology; clear research question and PICOs model. The search terms are inadequate. Should have included more terms like incremental cost, incremental effects, ICER, QALYs, DALYs etc. Line 116-118: This information is not needed. Line 173-174: The authors have used JBI checklist for systematic review of economic evaluations however, they should have also used the GRADE tool to determine the certainty in evidence. Results: Line 229-230: Amongst the included studies, only a single study has reported outcome in the form of DALYs which is an outcome measure for cost-utility analysis. The authors could have excluded this study to reduce heterogeneity. Else they could have expanded their search to include QALYs, DALYs, and cost-utility analysis along with some other relevant terms to yield more comprehensive results. Table 3 provides all the information regarding cost of interventions as well as control, however, there is no clear information in text as to what exactly the control group was because there is some cost that is mentioned for control group as well in table 3. The authors did not report whether the included studies were reported according to health economics guidelines e.g., CHEERS checklist. Discussion: Too lengthy discussion Findings should be summarized, too much extra information. Line 302-314: Too wordy, can be made brief. Line 322-330: Too much information about a cost-effectiveness analysis, can be removed from here and included in the introduction. Line 427-432: Not required. Limitations: The authors could have mentioned another limitation as the majority of included studies were model-based hence the results could not be adequately generalizable; this is an inherent limitation of cost-effectiveness analysis based on analytic modeling approach as the data on which the model is built may be biased in some way or the other thus, limiting the generalizability of the findings. ****** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: Yes: Nighat Naved ******** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. Attachments Attachment Submitted filename: Review PLOSone.docx https://doi.org/10.1371/journal.pone.0284518.r001
Revision 1
28 Mar 2023 Author Response Reviewer #1 1. The title is misleading, you do not do a CEA, you do a SR on other CEA’s = The title changed to “Systematic review on Cost-Effectiveness Analysis of School-Based Oral Health Promotion Program (Page 1 / Line 1) 2. In several parts of the study, the English language needs to be improved = The manuscript has been sent to the proofreading company for an English brush-up. 3. The setting of the oral health program is not presented or discussed in detail, even though that is highly influential when performing promotive and preventive programs. School systems in different countries tend to differ. = Added in discussion section (Page 22) 4. Negative cost-effectiveness ratios should not be presented in values as such ratios can’t be interpreted (more than saying that it is dominant). Similar, the use of terms such as “most-effective” is not correct, which you use in your research question. = All negative cost-effectiveness ratio has been reworded into cost-saving tendency. Research questions have been changed into “What kind of school-based oral health promotive and preventive program targeted elementary school children are categorized as the cost-saving program? (Page 2 & 14 / Line 36 & 256 - Page 4 / Line 84) 5. You state that you assess the risk of bias, but I can’t find any presentation of this. Your study quality assessment is more of an indicator whether certain aspects are covered in the analysis, rather than analysing the risk-of-bias of those aspects within the study. = Limitation and applicability assessment added. (Page 10-11) 6. Some parts of the result sections belong to the discussion. = Revised 7. I was surprised that only five studies were included. I think you need to present the reference to the 72 excluded full text studies in appendix and present what main criteria they failed to fulfil. = Appendix 1 attached. 8. In my opinion the use of the dominance ranking matrix analysis does not provide any additional valuable information in this systematic review. Hence, I think Table 4 and 5 can be omitted, this information is already presented in Table 3. = Table 4 and 5 were omitted 9. There are unnecessary part in the discussion, you don’t have to explain what a CEA and an ICER is or how to interpret it, that should (if needed) have been presented in the introduction. However, you need to compare your findings with other studies. There are several systematic reviews in child oral health programs that should have been discussed here (even though they have not used school as a strict criterion for the programs), for example by Hettiarachchi et al 2018, Frahait et al 2019, Eow et al 2019, Davidson et al 2021. = Discussion revised (Page 22) 10. You write that there are several limitations with the study, but you only mention two such ones (variety of health indicators and your searching keywords). I think there are other important weakness such as lack of risk-of-bias analysis, presentation of the settings, potential missing studies etc. = Revised. Generalizability, studies’ risk of bias, the use of health economic guidelines, and missing of potential studies were added as study limitations. (Page 30) Reviewer #2 Major points: 1. The manuscript is quite wordy and should be significantly reduced in length. = Word reduced. 2. There are several formatting errors, especially the referencing. References should be cited at the end of the statement and after full stop. = Revised. 3. Try to write the manuscript in third-person past tense and get it reviewed by someone proficient in English writing. = Done Introduction 4. Very well written; writing is supported by evidence 5. Line 48-51: There is redundancy, can be rephrased to a single statement. = Line revised. (Page 3 / Line 51-52) 6. Line 64: Reference should never be in the middle of statement. = Revised (Page 3 / Line 66) 7. Some details about cost-effectiveness analysis and its utility can be added here in the introduction. = Added. (Page 4 / Line 69) Methods 8. Fairly adequate description of the methodology; clear research question and PICOs model. 9. The search terms are inadequate. Should have included more terms like incremental cost, incremental effects, ICER, QALYs, DALYs, etc. = In the search keywords (for every database), we included “Incremental cost”, “Incremental effects”, and “ICER”. We did not include DALYs and QALYs to avoid getting more irrelevant articles. (Page 6) 10. Line 116-118: This information is not needed = Information removed. (Page 6 / Line 120) 11 Line 173-174: The authors have used JBI checklist for systematic review of economic evaluations however, they should have also used the GRADE tool to determine the certainty in evidence. = Limitation and applicability assessment added (using NICE-GRADE assessment tools) (Page 10-11) Result 12. Line 229-230 Amongst the included studies, only a single study has reported outcomes in the form of DALYs which is an outcome measure for cost-utility analysis. The authors could have excluded this study to reduce heterogeneity. Else, they could have expanded their search to include QALYs, DALYs, and cost-utility analysis along with some other relevant terms to yield more comprehensive results. = We decided to include this study with the agreement of DALYs as a health indicator in mind that was needed to measure the incremental effect. Moreover, the study provides incremental cost, as well as incremental effect, that is needed to measure the cost-effectiveness ratio. 13. Table 3 provides all the information regarding cost of interventions as well as control, however, there is no clear information in text as to what exactly the control group was because there is some cost that is mentioned for control group as well in table 3. = The control group was explained in the eligibility criteria. Also in page 15 (Page 5 / Line 103)(Page 15/ Line 256) 14. The author did not report whether the included studies were reported according to health economic guidelines e.g., CHEERS checklist. = Information added in Table 2 (Page 13) Discussion 15. Too lengthy discussion = Word reduced 16. Findings should be summarized, too much extra information. = Revised 17. Line 302-314: Too wordy, can be made brief. = Revised 18. Line 322-330: Too much information about cost-effectiveness analysis, can be removed from here and included in the introduction. = Revised 19. Line 427-432: Not required. = Omitted Limitations 20. The authors could have mentioned another limitation as the majority of included studies were model-based hence the results could not be adequately generalizable; this is an inherent limitation of cost-effectiveness analysis based on analytic modelling approach as the data on which the model is built may be biased in some way or other thus, limiting the generalizability of the findings. = Revised. Generalizability, studies’ risk of bias, the use of health economic guidelines, and missing of potential studies were added as study limitations (Page 30) Attachments Attachment Submitted filename: Reply to Reviewers.docx https://doi.org/10.1371/journal.pone.0284518.r002
3 Apr 2023 Decision Letter - Fahad Umer, Editor Systematic review on Cost-Effectiveness Analysis of School-Based Oral Health Promotion Program PONE-D-23-00755R1 Dear Dr.Thinni Nurul Rochmah , We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Fahad Umer Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0284518.r003
Formally Accepted
11 Apr 2023 Acceptance Letter - Fahad Umer, Editor PONE-D-23-00755R1 Systematic review on Cost-Effectiveness Analysis of School-Based Oral Health Promotion Program Dear Dr. Rochmah: I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department. If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org. If we can help with anything else, please email us at plosone@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. Fahad Umer Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0284518.r004

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .