PONE-D-24-08864Investigating Ethical Tradeoffs in Crisis Standards of Care through Simulation of Ventilator Allocation ProtocolsPLOS ONE

Dear Dr. Herington,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Jun 13 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Ronaldo Go, MD

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at 

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and 

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Thank you for stating the following in the Acknowledgments Section of your manuscript: 

"The authors would like to acknowledge the efforts of Data Science Masters’ students - Derek Caramella, Ezgi Siir Kibris, Nefle Nesli Oruc, and Walter Burnett - for assisting with initial exploration of the dataset under the supervision of JH. Final data acquisition, cleaning, analysis and simulation work was conducted by JH. The authors would like to thank Michael Nabozny, William Parker, Lainie Ross and the members of the URMC African American Patient and Family Advisory Council for comments on earlier versions of the project. The project described in this publication was partially supported by the University of Rochester CTSA award number UL1 TR002001 from the National Center for Advancing Translational Sciences of the National Institutes of Health. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health."

We note that you have provided funding information that is not currently declared in your Funding Statement. However, funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form. 

Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows: 

"JH, JHW, AC, and MZ received funding through the University of Rochester CTSA award number UL1 TR002001 from the National Center for Advancing Translational Sciences of the National Institutes of Health. The funder played no role in the study design, data collection and analysis, decision to publish or preparation of the manuscript."

Please include your amended statements within your cover letter; we will change the online submission form on your behalf.

3. Please include your full ethics statement in the ‘Methods’ section of your manuscript file. In your statement, please include the full name of the IRB or ethics committee who approved or waived your study, as well as whether or not you obtained informed written or verbal consent. If consent was waived for your study, please include this information in your statement as well. 

4. Please upload a copy of Supporting Information Figure/Table/etc. Fig S7-S10 and Table S8-S11 which you refer to in your text on page 19.

5. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments:

Dear Dr. Jonathan, Herington,

Thank you for allowing us to review your manuscript: PONE-D-24-08864 "Investigating Ethical Tradeoffs in Crisis Standards of Care through Simulation of Ventilator Allocation Protocols." We require minor revisions and please submit as soon as possible. If you are not able to see the reviewers' comments, do not hesitate to contact me.

Best regards,

Ronaldo Go MD

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: In this excellent and well-written study, Herrington et. al perform a monte carlo simulation of ventilator allocation under crisis standards of care. They compare the results of various published triage protocols, finding that age-based triage vastly outperforms SOFA-tiers in terms of lives and life-years saved. The analysis adds significantly to the prior literature and has important findings for refining crisis standards of care for future disasters. The simulation approach is very clever in handling the various capacity levels and the visual representations of the simulation results are outstanding.

I believe the following comments will strengthen the manuscript:

1. More details on the SOFA score calculation should be added to the methods. Were the SOFA score components calculated automatically by the EHR software or by the investigators from the underlying EHR data? Was the original 1997 definition of SOFA used?

2. When exactly was the SOFA score calculated? Was the worst value of each SOFA component in the first 24 hours following the initiation of ventilation used to calculate the total SOFA score? The discussion says the study only used “SOFA scores collected at point of intubation”, which implies this calculation was happening using data from the 24 hours prior to intubation. This needs to be clarified in the methods.

3. 320 patients were excluded because they lacked lab results for SOFA, but earlier missing SOFA values were “assumed to be normal”.

4. What does “an unclear pattern of admissions and intubations (n = 332)” mean exactly?

5. I would not recommend referring to the SOFA plus comorbidity with age tie-breaker as “the Bhavani protocol” throughout the paper. This protocol was not explicitly endorsed by Bhavani or the other authors of the manuscript and seems to have been designed to represent several different state plans (e.g., Maryland, Pennsylvania).

6. What was the etiology of respiratory failure requiring mechanical ventilation in the 66% of patients who did not have COVID? This information should be added to Table 1.

7. Related to the previous comment, the patient population during a crisis scenario would likely differ from the general population who required mechanical ventilation from 4/2020- 5/2021. For example, during an overwhelming COVID-19 surge, patients who required mechanical ventilation following an elective surgery would not be included and the cohort would likely be mostly COVID +. I recommend the authors run sensitivity analyses with sub-populations for specific crisis scenarios (e.g. 90% COVID + and no elective surgery patients).

8. Did the authors race-adjust life-expectancy and life-years in Table 4? I would argue this choice normalizes racial disparities in life-expectancy which hopefully will improve over time with efforts to address structural inequity in healthcare. So I recommend these statistics be re-calculated for age-alone.

9. The updated Colorado guidelines currently include a correction to the renal SOFA score component in an attempt to debias SOFA. I recommended adding this to the simulation results.

Minor:

In the abstract, would revise to the following: For a capacity of 1 bed per 2 patients, ranking by age bands saves approximately 28.7 lives and 3408 life-years per thousand patients, while ranking by Sequential Organ Failure Assessment

20 (SOFA) bands saved 13.2 lives and 416 life-years per patient (**statistical testing**)

Reviewer #2: I congratulate you on a well done study with important findings. AS pointed out, in crisis mode, many institutions and governing bodies have suggested using allocation algorithms employing SOFA scores. Although simple to calculate and readily available, the SOFA score does not contain parameters for age. As your paper has pointed out, age is clearly a very important factor to consider in saving lives and life years. Also as you pointed out in the discussion, there are many political and legal issues with using age concerning discrimination.

Although the most optimal way to stratify patients during periods of CSC remains unclear, this paper advocates that SOFA alone may not be enough, and that age should be included in some way. It's important to note that the Colorado system utilizes the Charlson comorbidity Index, and this index contains age as one of the components in its calculation with significant weight.

I also had a question about the relatively low percentage of COVID cases. During this crisis time frame, most hospitals were overwhelmed with COVID cases, typically shutting down ORs except for emergency cases, etc. It seems odd that only 34% of the critically ill cases would be secondary to COVID. The overall survival between COVID and non COVID was similar, but was there a further subgroup analysis?

I appreciate that breaking down SOFA scores into tiers or bands has its drawbacks and is likely a significant limitation with the NY score using only three bands. This will also lead to more tiebreaks which are done by lottery.

Lastly I agree there seems to be a population bias which is heavily affecting the racial outcomes. This part of the paper was difficult to explain. I encourage you to do the regional analysis as you suggested.

One small point.

Methods section Line 64: You state the implementation of five proposed ventilator allocation protocols: You then describe six protocols.

Reviewer #3: This study evaluates 5 proposed CSC (Crisis Standards of Care) protocols for rationing ventilators during the COVID19 pandemic for adults that required a ventilator between 4/2020 and 5/2021 in a Monte Carlo simulation model of ventilator allocation. The strengths of this study include the inclusion of non-COVID patients with respiratory failure requiring mechanical ventilation proving a more realistic dataset compared to other studies.

The limitations are well documented by the authors and include that it is a single center study, the non-Hispanic black and Hispanic patients are a much smaller sample than the white patients, many non-Hispanic black and Hispanic patients may have been triaged to a different hospital within the same area, and there was no accounting for DNR code status.

Similar to other studies are the findings that allocation protocol favoring younger age saved the most lives and added the most “life-years” and SOFA based protocol saved fewer lives than age allocated. What is different from other studies is the observation that whites experienced significantly lower allocation rates, lives saved, and life-years saved with the implementation of all the protocols except two: NY SOFA and lottery.

Since age has a strong relationship with survival from COVID, and the black and Hispanic patients were younger, this may partly explain why they fared better than the older white patients. But as pointed out bias in the distribution of patients between two different metropolitan hospitals that serve particular patient populations may also result in these differences.

The authors are to be commended for their thorough analytic approach, clear presentation of the statistics, thoughtful interpretation of the results, and the provided link to the code used to perform the analysis.

Typo:

“agre” = age

Table S8: Survival by Agre Group at 50% capacity.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: William F Parker

Reviewer #2: No

Reviewer #3: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Investigating Ethical Tradeoffs in Crisis Standards of Care through Simulation of Ventilator Allocation Protocols

PONE-D-24-08864R1

Dear Dr. Herington,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Ronaldo Go, MD

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: (No Response)

Reviewer #2: I believe this should be accepted. It is well written and a well performed analysis. I believe the conclusions are valid.

All of my concerns have been addressed. I look forward to seeing this article published. Thank you for allowing me to review this article.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: William F Parker

Reviewer #2: Yes: Keith M. Rose, MD

**********