PONE-D-23-27466A model-based cost-utility analysis of an automated notification system for deteriorating patients on general wardsPLOS ONE

Dear Dr. Subbe,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Nov 25 2023 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Dominic Luke Thorrington, PhD

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at 

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and 

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Note from Emily Chenette, Editor in Chief of PLOS ONE, and Iain Hrynaszkiewicz, Director of Open Research Solutions at PLOS: Did you know that depositing data in a repository is associated with up to a 25% citation advantage (https://doi.org/10.1371/journal.pone.0230416)? If you’ve not already done so, consider depositing your raw data in a repository to ensure your work is read, appreciated and cited by the largest possible audience. You’ll also earn an Accessible Data icon on your published paper if you deposit your data in any participating repository (https://plos.org/open-science/open-data/#accessible-data).

3. Thank you for stating the following financial disclosure: 

 "Yes - the study was funded by Philips Healthcare. Bangor University received funding from Betsi Cadwaladr University Health Board, Wales, UK to complete this research, via a Research Service subcontract of “Specific Research Plan for Economic Evaluation of Philips IntelliVue Guardian Solution (IGS)” incorporated into the Observational Field Test/Post Market Study Agreement between Philips Medizin Systeme Böblingen GmbH and Betsi Cadwaladr University Local Health Board."  

Please state what role the funders took in the study.  If the funders had no role, please state: ""The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript."" 

If this statement is not correct you must amend it as needed. 

Please include this amended Role of Funder statement in your cover letter; we will change the online submission form on your behalf.

Additional Editor Comments:

Thank you for your submission to PLOS One and for your very interesting manuscript.

Based on my own reading of the manuscript, but more so from the comments from the second reviewer, your manuscript requires major revision before it can be published. The second reviewer has included several detailed comments that require addressing, but should improve the manuscript.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: No

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The method is good but I do not agree with this topic. In a hospital with deteriorating patients both deployment of an electronic automated advisory vital signs and routine monitoring are needed. So it is crystal clear and does not need a study. You can not replace them but to have both. I can not accept this topic at all.

Reviewer #2: This paper presents a policy-relevant cost-effectiveness investigation utilizing peer-reviewed inputs and generally standard approaches. The analysis benefits both from a well-performed sensitivity analysis that allows for joint fluctuation in all parameters and from an array of sub-group analyses. The paper is well organized and the presentation is generally clear. Overall this is a solid analysis.

This paper does contain at least one key flaw, but one that should be fixable in a fairly straightforward way. For all its elegance in probabilistically sampling an array of input parameters for its sensitivity analysis, the fundamental costing of the intervention is misconstrued in a manner that underestimates it. The authors estimate a per-patient use cost by taking the full annual cost (purchase price divided by useful life), and apportioning to an average patient the average proportion of a year a patient utilizes the monitoring system. This is incorrect in that it results in treating the time that the system is unused as costless, as though the NHS could only cover a percentage of the purchase price equal only to the percent of the year it will be actively used. Even as a distal payer, the full cost of the system will be passed to the NHS, and the true cost per patient will be very simply the annual cost divided by annual patients. Estimating this way will raise the estimated intervention cost per patient, and every ICER reliant upon it. However, the change will not alter findings to an extent that will alter the fundamental finding that the system will be well with in cost-effectiveness thresholds.

There is also another aspect of this paper that may or may not be correct, and discussion (if not revision) is warranted. It is very noteworthy that the (adjusted) difference in point estimates of the mortality between intervention and control (as presented in Table 2 and Table S9, roughly -.001) in are actually quite small and do not match the adjusted mortality reductions discussed in the Clinical Background section and reported in reference 14 (roughly -.016, AOR ~.079). Why is this? This difference certainly has a large influence on the estimate of the resulting effectiveness inputs. Discussion and possibly revision are warranted. This difference may be underselling the cost-effectiveness of the system.

These being my main concerns with the manuscript, a handful of other areas would benefit from further attention:

- On line 49, for clarity it would help if the phrasing were "remains cost-effective at 100% 'usage' " lest some readers infer a meaning of "remains cost effective at 100% probability."

- The "deterioration" which triggers a notification should be more clearly defined in the clinical background section. I found myself turning to the underlying investigation (ref 14) to understand this. It was otherwise somewhat unclear at first read if a notification could meaningfully precede "events" and forestall them or simply more vigorously rallied clinical staff when an event was already occurring.

- On line 141 refers readers to S1 to understand how QALYs are estimated in the model. The example in S1 is well taken, However, the reader also needs to be referred to Table 2 for the sources of QALYs, OR they need to be also reported in S1. At least a sentence or two on how the authors searched for and/or selected these as their sources is also warranted. These are very key inputs.

- Regarding lifetime resource use as discussed briefly in lines 146-150, this deserves more explanation. I see estimates with sources in Table 2, and the reader should be pointed there. Moreover, if the utilized estimates are directly reported in those sources, that should be mentioned in this paragraph. If the authors did analysis with those sources to arrive at the estimates in Table 2, that analysis should be described here.

- Regarding the equations in lines 158-160, it is appreciated that the authors use as their main analysis a conservative 5-yr useful product life, and leave longer estimates to sensitivity analyses. For completeness, if training costs are a regular part of initiating use of this technology, that should also be included, or their omission mentioned as a limitation. Similarly, if maintenance costs are a regular cost of utilizing this technology, they should also be incorporated or their omission should be mentioned as a limitation.

- Regarding the footnotes in Table 1 and the related S2-5: The weighted average should mention what weights; I'm inferring "frequency weighted average," correct? Relatedly, in S5, is "activity" synonymous with "frequency"? And is S5 and input into S4? In which case I would re-order them.

- Lines 185 - 187 is the only place in the main text where discounting is discussed, and only with reference to the inpatient stays. Is discounting performed for lifetime (post-stay) costs and/or QALYs?

- the Long-term costs section should at some point refer the reader clearly to Table 2.

- Within Table 2, the "Distribution" column should also report either the relevant confidence interval or variance parameter around the point estimate in the 'point-estimate' column.

- In Table 4, I note that though Quality-adjusted life expectancy is higher in the Intervention group, raw life expectancy is reported at about .014 life years shorter. Is this accurate? Also, why is this the only row in Table 4 to not have 95% central ranges? They should be included in this row as with the others if available.

- The note in Table S9 states the adjusted probabilities are adjusted for "age, sex, gender...." I wanted to check that it's accurate that the adjustment included both sex and gender distinctly construed? Or was it just one of them?

These detailed comments notwithstanding, the authors should be commended on a solid investigation, and I would expect that with revision this paper will reach a publishable form.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Dr. Seyyed Mostafa Hakimzadeh

Reviewer #2: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PONE-D-23-27466R1A model-based cost-utility analysis of an automated notification system for deteriorating patients on general wardsPLOS ONE

Dear Dr. Subbe,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Feb 26 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Dominic Luke Thorrington, PhD

Academic Editor

PLOS ONE

Additional Editor Comments:

One of the previous reviewers has highlighted a minor issue that needs to be dealt with, whilst a new reviewer has left several major comments on the manuscript:

 - a more detailed description of the model is required

 - some cost items in the economic model are missing, which the reviewer highlights as a serious flaw

 - additional information is required for the VITAL II study, given its importance to this manuscript

 - some additional information is needed to describe the general modelling process for the reader In addition to the bullet points above, I invite you to read the detailed feedback from both reviewers before proceeding with amendments to your manuscript.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #2: (No Response)

Reviewer #3: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #2: Yes

Reviewer #3: No

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2: Yes

Reviewer #3: No

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: Yes

Reviewer #3: No

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #2: Yes

Reviewer #3: No

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #2: This revised manuscript has made excellent progress in addressing my previous comments, most of which have been thoroughly cleared up. I hold to but one sticking point on the per-patient costs as discussed in my primary comment in the original description.

The authors have now added clarity that their calculations assume an annual patient number based on operating at full ward capacity. Because this assumption is now explicit, this tact is acceptable so long as they add somewhere in the discussion or limitations sections that this assumption results in an optimistic bound for this parameter's effect on the cost effectiveness of the intervention. However, having re-checked reference 14, it is also suggested that the authors note that the wards did run at near capacity during the intervention. [To wit, 2263 patients were served in in the ~25 ward months as opposed to the expected 357/8.62*54 = 2286 patients annually across the two wards]. Alternately, the authors may re-run their analyses with a more empirical annual number of patients, but that would seem a fair amount of effort for a minor shift in precision, and I'm not requiring that so long as the optimistic effect of their 100% capacity assumption is explicitly noted.

Otherwise, this is a strong analysis that I expect to approve for publication pending this minor revision.

Reviewer #3: This is an interesting paper which reports the cost-effectiveness analysis of automated notification system for deteriorating patients on general wards in the UK. While the topic is worthy of study, the current paper does not provide sufficient detail on the methods used, I have concerns over some of the approaches taken and additional detail is required throughout. Please find some specific comments below.

Major comments:

1) A more detailed description of the model is required, including an improved model schematic. For example, it is unclear how repeat events have been addressed in the model.

2) The study does not appear to include any costs assocaited with responding to a notification system- this appears to be a significant oversight.

3) A more detailed description of the VITAL II Study is required (e.g. patients included, types of ward etc)

4) More details on cost-effectiveness methods need to be provided for a clinical audience (e.g. what is a QALY, a threshold, discounting, cost-effectiveness decision rules etc)

further comments

1) The methods section of abstract should be expanded- for example, including scenario analyses

2) The authors should complete a CHEERS checklist to ensure that the economic evaluation is appropriately reported

3) WHow is NEWS measured? (Note first time it is used as just an acronym with this not defined until line 131)

4) Why did you restict patients in VITAL to those with NEWS of 15?

5) For what purpose was a review of economic evaluations conducted? No real mention of how if at all these were used to inform this study.

6) Need to provide details on the lifetime costs used (I don't see these reported in Table 1 as stated)

7) I don't think providing the distributions for the PSA in Table 1 is particularly useful here (particularly the numbers used). Would be better to report something on the parameter uncertainty (e.g. standard error)

8) Following on- why use arbitrary level of parameter uncertainty of 0.2 of mean? This seems unnecessaary for many of the parameters used where you would be able to estimate the actual uncertainty

9) Annuitization should be used to calculate the cost of the product per patient

10) More justication is required for the 100% ward capacity assumption

11

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: No

Reviewer #3: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

A model-based cost-utility analysis of an automated notification system for deteriorating patients on general wards

PONE-D-23-27466R2

Dear Dr. Subbe,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at Editorial Manager® , click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Dominic Luke Thorrington, PhD

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Thank you for responding to the reviewers' recommendations and comments on the first revision of the manuscript. The manuscript is now much improved.

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #3: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #3: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #3: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #3: The authors have responded in a satisfactory manner to all of my comments. I have no further comments to add/

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #3: No

**********