Efficacy, cost-minimization, and budget impact of a personalized discharge letter for basal cell carcinoma patients to reduce low-value follow-up care

Sven van Egmond; Ella D. van Vliet; Marlies Wakkee; Loes M. Hollestein; Xavier G. L. V. Pouwels; Hendrik Koffijberg; Yesim Misirli; Rachel S. L. A. Bakkum; Maarten T. Bastiaens; Nicole A. Kukutsch; Albert J. Oosting; Elsemieke I. Plasmeijer; Annik van Rengen; Kees-Peter de Roos; Tamar E. C. Nijsten; Esther de Vries; Esther W. de Bekker-Grob

doi:10.1371/journal.pone.0260978

Peer Review History

Original SubmissionJuly 28, 2021
30 Sep 2021 Decision Letter - János G. Pitter, Editor PONE-D-21-24499Efficacy, cost-utility, and budget impact of a personalized discharge letter for basal cell carcinoma patients to reduce low-value follow-up carePLOS ONE Dear Dr. van Egmond, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please find below the issues identified by the peer reviewers and compose the corresponding minor revision of the manuscript. It is acknowledged that the template personalized discharge letter is already included in the Supporting information. During your revision, please also take into consideration the following editorial comments: 1) A key modelling assumption was that the introduction of personalized discharge letter had no effect on clinical outcomes and patient QALYs (i.e., the "low-value" follow-up visits were assumed to have zero clinical value in the model). Hence, it is recommended to discuss how evidence-based this assumption was. Given that no QALY difference was allowed across intervention and control cohorts by the model structure (except for stochastic difference in the probabilistic sensitivity analysis), the developed model is considered to be a cost minimization model, rather than cost-utility model: please consider adjusting the manuscript title accordingly. 2) For better reproducibility, please make an explicit statement about the applied appointment probabilities in Years 2-9 in the intervention cohort in Table S3.2. 3) In line with the comments of Reviewer 1, please include the verbatim recommendations on follow-up visit frequency of the corresponding guidelines for the two cohorts, i.e., before and after the implementation of the personalized discharge letter approach. This information will help readers to interpret any possible interference of guideline revision with the study intervention. Please submit your revised manuscript by Nov 14 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. We look forward to receiving your revised manuscript. Kind regards, János G. Pitter, MD, PhD Academic Editor PLOS ONE Journal Requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. Thank you for stating the following in the Acknowledgments/ Funding Section of your manuscript: This project was funded by Citrienfonds (Dutch Ministry of Health, Welfare and Sport) and VGZ (Health insurance company). The funders were not involved in study design, data collection, data analysis, and manuscript preparation. Please note that funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form. Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows: This project was funded by Citrienfonds (Dutch Ministry of Health, Welfare and Sport, https://www.citrienfonds.nl/, received by TN) and VGZ (Health insurance company, https://www.vgz.nl/, received by EdV). The funders were not involved in study design, data collection, data analysis, and manuscript preparation. Please include your amended statements within your cover letter; we will change the online submission form on your behalf. 3. We note that Appendix S1 includes an image of a participant. As per the PLOS ONE policy (http://journals.plos.org/plosone/s/submission-guidelines#loc-human-subjects-research) on papers that include identifying, or potentially identifying, information, the individual(s) or parent(s)/guardian(s) must be informed of the terms of the PLOS open-access (CC-BY) license and provide specific permission for publication of these details under the terms of this license. Please download the Consent Form for Publication in a PLOS Journal (http://journals.plos.org/plosone/s/file?id=8ce6/plos-consent-form-english.pdf). The signed consent form should not be submitted with the manuscript, but should be securely filed in the individual's case notes. Please amend the methods section and ethics statement of the manuscript to explicitly state that the patient/participant has provided consent for publication: “The individual in this manuscript has given written informed consent (as outlined in PLOS consent form) to publish these case details”. If you are unable to obtain consent from the subject of the photograph, you will need to remove the figure and any other textual identifying information or case descriptions for this individual. 4. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Partly Reviewer #2: Yes ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: I Don't Know Reviewer #2: Yes ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ****** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: Thank you for asking me to review this paper. This paper looks at the use of a personalised software generated discharge letter to reduce follow up visits following a first BCC. Prevalence is misused as a term- 80% of Caucasians don’t have a BCC. They don’t define a low risk BCC- or justify 50% of BCCs in their cohort being low risk. They should be clear if their group pf BCCS were all completely excised and if so what parameters they use to define that – e.g mohs/ 1mm or more histological clearance at all margins on standard surgical excision with curative intent. Software related costs are mentioned- they should state this is one way of delivering this, but as with other patient letters, software is not mandatory. The introduction of new guidelines could have had a significant impact on the behaviour of and while the authors are right to mention is, it is a major risk that this led to some or most of the reduction in visits. They should state how follow up guidance changed to be clear on what the impact may have been This paper demonstrates well that follow up was reduced coinciding with introduction of a personalised discharge letter. They explore that there is potentially a financial disincentive for clinicians to discharge a patient at one follow up visits. It is common practice in the UK for further visits to simply not be funded and it would be useful to discuss the pros and cons of excessive follow ups, from a patient and clinician and health care economy perspective. Reviewer #2: Thank you for the nice report. It is nicely written. Correct use of statistics and well explained. The article adds value to current literature. What is written in the personalized discharge letter? An example in het Supplementary would be usefull. What factors in the letter make it 'personal'/ what are the differences between the letters and based on which characteristics of the patient? I have no further comments. ******** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0260978.r001
Revision 1
14 Nov 2021 Author Response Reviewer #1: Thank you for asking me to review this paper. This paper looks at the use of a personalised software generated discharge letter to reduce follow up visits following a first BCC. 1) Prevalence is misused as a term- 80% of Caucasians don’t have a BCC. The term was indeed misused and has been adjusted accordingly. (p.3, line 76) 2) They don’t define a low risk BCC- or justify 50% of BCCs in their cohort being low risk. Low-risk BCCs are defined as primary BCC, < 2 cm, located outside the H-zone, with a nodular or superficial subtype according to the Dutch BCC guidelines. This has been added to the manuscript. (p.3, lines 86-87) Regarding the justification of 50% of BCCs; This number was extrapolated from the previous work of the authors (de Vries et al.), as cited in the manuscript. 3) They should be clear if their group pf BCCS were all completely excised and if so what parameters they use to define that – e.g mohs/ 1mm or more histological clearance at all margins on standard surgical excision with curative intent. We indeed do not provide the numbers of completely excised BCCs. This has not been registered for all patients. We acknowledge that this is a limitation of our study and have added it as such to the manuscript. (p.16, lines 354-355) 4) Software related costs are mentioned- they should state this is one way of delivering this, but as with other patient letters, software is not mandatory. By software costs we intended the costs needed to implement the feature in the current electronic health record system the hospital already uses. We have clarified this in p.7, lines 198-199. 5) The introduction of new guidelines could have had a significant impact on the behaviour of and while the authors are right to mention is, it is a major risk that this led to some or most of the reduction in visits. They should state how follow up guidance changed to be clear on what the impact may have been We agree with the reviewer that clarifying this is valuable for the interpretation of the discussion. We have clarified in the discussion that this is the first skin cancer guideline for GPs (p.15, line 340) and that the guidelines states that after complete excision and a post-treatment evaluation after 3 months, further monitoring is not necessary (p.15, lines 343-344). However, we would like to emphasize that there have not been any changes in the dermatologist guidelines regarding BCC follow-up care, and the dermatologist is in control of the patients’ follow-up schedule (p16, lines 349-350) 6) This paper demonstrates well that follow up was reduced coinciding with introduction of a personalised discharge letter. They explore that there is potentially a financial disincentive for clinicians to discharge a patient at one follow up visits. It is common practice in the UK for further visits to simply not be funded and it would be useful to discuss the pros and cons of excessive follow ups, from a patient and clinician and health care economy perspective. We agree with the reviewer that a further exploration on the effect of low-value care is an interesting direction. We have chosen to focus on the possible implementation effects of this particular intervention on all affected parties, ranging from employers to clinicians and insurers. We have added an extra perspective to the manuscript (p.15, line 335-337). Reviewer #2: Thank you for the nice report. It is nicely written. Correct use of statistics and well explained. The article adds value to current literature. 1) What is written in the personalized discharge letter? An example in het Supplementary would be useful. What factors in the letter make it 'personal'/ what are the differences between the letters and based on which characteristics of the patient? I have no further comments. We are sorry to notice that there might have been an administrative issue during the submittal process. An example of the personalized letter was already added in the supplementary. We hope that this will find you well in the current rebuttal. Editorial comments 1) A key modelling assumption was that the introduction of personalized discharge letter had no effect on clinical outcomes and patient QALYs (i.e., the "low-value" follow-up visits were assumed to have zero clinical value in the model). Hence, it is recommended to discuss how evidence-based this assumption was. Given that no QALY difference was allowed across intervention and control cohorts by the model structure (except for stochastic difference in the probabilistic sensitivity analysis), the developed model is considered to be a cost minimization model, rather than cost-utility model: please consider adjusting the manuscript title accordingly. The effect of the personalized letter on the utility values was measured in the current study by administrating the EQ-5D-3L questionnaire. We agree with the editor that by having similar utility values in each group, the analysis is in fact a cost-minimization analysis. Extra clarification has been added to the manuscript (p. 7, line 180-182). We have also changed the title to “Efficacy, cost-minimization, and budget impact of a personalized discharge letter for basal cell carcinoma patients to reduce low-value follow-up care” 2) For better reproducibility, please make an explicit statement about the applied appointment probabilities in Years 2-9 in the intervention cohort in Table S3.2. We have improved the readability by starting with the control group and providing extra specification about the probabilities in the supplementary material. 3) In line with the comments of Reviewer 1, please include the verbatim recommendations on follow-up visit frequency of the corresponding guidelines for the two cohorts, i.e., before and after the implementation of the personalized discharge letter approach. This information will help readers to interpret any possible interference of guideline revision with the study intervention. We agree with both the reviewer and editor that further specification was needed regarding the possible guideline interference. This has been adjusted accordingly as stated in response to reviewer 1, comment 5. Attachments Attachment Submitted filename: Response to reviewers.docx https://doi.org/10.1371/journal.pone.0260978.r002
22 Nov 2021 Decision Letter - János G. Pitter, Editor Efficacy, cost-minimization, and budget impact of a personalized discharge letter for basal cell carcinoma patients to reduce low-value follow-up care PONE-D-21-24499R1 Dear Dr. van Egmond, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, János G. Pitter, MD, PhD Academic Editor PLOS ONE Additional Editor Comments (optional): Reviewers' comments: https://doi.org/10.1371/journal.pone.0260978.r003
Formally Accepted
12 Jan 2022 Acceptance Letter - János G. Pitter, Editor PONE-D-21-24499R1 Efficacy, cost-minimization, and budget impact of a personalized discharge letter for basal cell carcinoma patients to reduce low-value follow-up care Dear Dr. van Egmond: I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department. If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org. If we can help with anything else, please email us at plosone@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. János G. Pitter Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0260978.r004

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .