PONE-D-24-11345Study Protocol for a Randomized Controlled Trial: Integrating the ‘Time-limited Trial’ in the Emergency DepartmentPLOS ONE

Dear Dr. Hashimoto,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Overall, the protocol for implementing time limited trials in the emergency department is interesting.  I commend you on exploring tools to improve communication with families in these difficult clinical situations.  The reviewers raised several important critiques which are highlighted below.  I ask you to pay particular attention to the following issues:

Reviewer 1 appropriately comments that initiating time limited trials in the ED may work against prognostic accuracy and whether it may create communication discordance with ICU physicians.  This critique relates to a larger question about this protocol, namely, what are the advantages and disadvantages of implementing time limited trials in the ED (compared to ICU)?  Having TLTs initiated by ICU physicians would have the advantage of communication continuity, perhaps better rapport as there will generally be more time for family meetings in the ICU, availability of ICU nurses and other multi-disciplinary services, among numerous other factors.  One can argue that ICU physicians may also have more experience with the trajectory of critical illness and may be able to provide more detailed information on prognosis/hospital course.  It would significantly aid the rationale for the protocol to be more clear about why initiating TLTs in EDs is an advantage (compared to initiating in ICU) and potential drawbacks to doing it this way.  In addition, I would consider incorporating ICU physicians as stakeholders in the acceptability assessments.  The ICU physicians will be continuing the additional communications and shared decision-making, so including them seems crucial.

Reviewer 2 comments on sample size and analytic plans are critical to address.  In addition, it would be helpful to have more details on how the numerous assessments will be conducted.  For example, will there be a member of the research team to collect data on family meetings daily both day and night shifts?  How will they know that a family meeting will be conducted by a trained ED physician?   Who will distribute and collect surveys that require follow up (Decision regret scale)? 

Please submit your revised manuscript by Nov 15 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Dong Wook Chang

Academic Editor

PLOS ONE

Journal requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Thank you for stating the following in the Acknowledgments Section of your manuscript:

“Dr. Ouchi is supported by the National Institute on Aging (K76AG064434)”

We note that you have provided funding information that is not currently declared in your Funding Statement. However, funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form.

Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows:

“The author(s) received no specific funding for this work.”

Please include your amended statements within your cover letter; we will change the online submission form on your behalf.

3. Thank you for stating the following in the Competing Interests

“Dr.Ouchi has received funding personally from Jolly Good, Inc (a virtual reality company) for consulting”

We note that one or more of the authors are employed by a commercial company: Jolly Good, Inc.

a. Please provide an amended Funding Statement declaring this commercial affiliation, as well as a statement regarding the Role of Funders in your study. If the funding organization did not play a role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript and only provided financial support in the form of authors' salaries and/or research materials, please review your statements relating to the author contributions, and ensure you have specifically and accurately indicated the role(s) that these authors had in your study. You can update author roles in the Author Contributions section of the online submission form.

Please also include the following statement within your amended Funding Statement.

“The funder provided support in the form of salaries for authors [insert relevant initials], but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The specific roles of these authors are articulated in the ‘author contributions’ section.”

If your commercial affiliation did play a role in your study, please state and explain this role within your updated Funding Statement.

b. Please also provide an updated Competing Interests Statement declaring this commercial affiliation along with any other relevant declarations relating to employment, consultancy, patents, products in development, or marketed products, etc. 

Within your Competing Interests Statement, please confirm that this commercial affiliation does not alter your adherence to all PLOS ONE policies on sharing data and materials by including the following statement: "This does not alter our adherence to  PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests) . If this adherence statement is not accurate and  there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared.

Please include both an updated Funding Statement and Competing Interests Statement in your cover letter. We will change the online submission form on your behalf.

4. When completing the data availability statement of the submission form, you indicated that you will make your data available on acceptance. We strongly recommend all authors decide on a data sharing plan before acceptance, as the process can be lengthy and hold up publication timelines. Please note that, though access restrictions are acceptable now, your entire data will need to be made freely accessible if your manuscript is accepted for publication. This policy applies to all data except where public deposition would breach compliance with the protocol approved by your research ethics board. If you are unable to adhere to our open data policy, please kindly revise your statement to explain your reasoning and we will seek the editor's input on an exemption. Please be assured that, once you have provided your new statement, the assessment of your exemption will not hold up the peer review process.

5. We note that the original protocol that you have uploaded as a Supporting Information file contains an institutional logo. As this logo is likely copyrighted, we ask that you please remove it from this file and upload an updated version upon resubmission.

6. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Comments to the Author

1. Does the manuscript provide a valid rationale for the proposed study, with clearly identified and justified research questions?

The research question outlined is expected to address a valid academic problem or topic and contribute to the base of knowledge in the field.

Reviewer #1: Partly

Reviewer #2: Yes

********** 

2. Is the protocol technically sound and planned in a manner that will lead to a meaningful outcome and allow testing the stated hypotheses?

The manuscript should describe the methods in sufficient detail to prevent undisclosed flexibility in the experimental procedure or analysis pipeline, including sufficient outcome-neutral conditions (e.g. necessary controls, absence of floor or ceiling effects) to test the proposed hypotheses and a statistical power analysis where applicable. As there may be aspects of the methodology and analysis which can only be refined once the work is undertaken, authors should outline potential assumptions and explicitly describe what aspects of the proposed analyses, if any, are exploratory.

Reviewer #1: Partly

Reviewer #2: Partly

********** 

3. Is the methodology feasible and described in sufficient detail to allow the work to be replicable?

Descriptions of methods and materials in the protocol should be reported in sufficient detail for another researcher to reproduce all experiments and analyses. The protocol should describe the appropriate controls, sample size calculations, and replication needed to ensure that the data are robust and reproducible.

Reviewer #1: Yes

Reviewer #2: Yes

********** 

4. Have the authors described where all data underlying the findings will be made available when the study is complete?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception, at the time of publication. The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

********** 

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

********** 

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above and, if applicable, provide comments about issues authors must address before this protocol can be accepted for publication. You may also include additional comments for the author, including concerns about research or publication ethics.

You may also provide optional suggestions and comments to authors that they might find helpful in planning their study.

(Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This is an interesting tool to evaluate but the overall goal of pursuing Time Limited Trials in the ED is not fully clear. The second sentence of the Discussion in the Abstract seems to be the most pertinent: "...the study intends to improve palliative care integration for seriously ill older adults in the ED and intensive care unit." The authors clearly understand the need for Goals of Care conversation and the role of Palliative Care within the ED. The introduction of time limited trials in the ED is challenging, as the clinical trajectory could be unclear early in the patient's presentation and the goals that are discussed and set forth may not align with the ICU providers' who will be responsible in carrying them out. If the main points that are to be discussed are overall patient wishes, this aligns more with the framework of a general goals of care conversation than a TLT. The authors need to delineate more specifically the exact points of discussion are to be and how these conversations are separate form a more rudimentary goals of care conversation.

Reviewer #2: My major concern

The authors indicated, "No previous two-arm trial of TLT have been conducted; thus, the effect size remains unknown. Given the primary purpose of conducting the RCT is to demonstrate the feasibility, no gold standard for sample size calculations exists. However, prior literature suggests that 15 to 20 participants per group are required to ensure the scientific validity of the pilot study results [45].” The authors then decided to select 40 participants with 20 each intervention and control group. The authors then went on to say that “In addition, the results of this pilot study will provide information on the difference between groups and the standard deviation necessary for sample size calculation during the future full-scale study.”

These statements are contradictory. Now my question is if the study is not powered enough to see the minimum detectable effect (MID), on what basis can we use the MID estimate from a pilot study to power future studies. I agree that pilot studies have been used to estimate effect sizes to determine the sample size needed for a larger-scale randomized controlled trial (RCT) or observational study. However, this practice has been challenged because pilot study samples are usually small and unrepresentative, as this study has demonstrated by interviewing a total of 40 clinicians (20 controls and 20 interventions). There was no power analyses to arrive at those numbers which means that estimates of parameters and their standard errors may be inaccurate, resulting in misleading power calculations for even larger-scale studies. So adding that the “ the results of this pilot study will provide information on the difference between groups and the standard deviation necessary for sample size calculation during the future full-scale study.” May be problematic. Because the focus is on feasibility, results of statistical tests are generally not informative for powering the main trial outcomes. Additionally, feasibility process outcomes may be poorly estimated because the study is not powered.

The study is not powered since there was no power analysis and authors decided to only do 40 observations, so on what basis will the authors employ all these rigorous statistical analyses (two-sample t-test, binomial test of proportion, Wilcoxon rank sum test) and even run regression models. Remember the standard error estimation or precision of point estimate is based on the sample size which has effect on the estimation of p-values. So if no power analysis was done, how would you standard errors, confidence interval and p-values be reliable?

Also, note that small sample sizes can result in an imbalance between arms or within subgroups that cannot be detected with pilot data or early on in studies. Most pilot studies with smaller sample size should not be used to estimate effect sizes, provide power calculations for statistical tests or perform exploratory analyses of intervention impact.

My suggestion: I think the authors should focus on the primary objective of conducting feasibility studies which is to assess the feasibility/acceptability of Integrating the ‘Time-limited Trial’ in the Emergency Department (the time it takes to conduct the TLTs by ED clinicians) and clinician and patient-reported acceptability of the TLT but without a more rigorous power analysis, the statistical analyses at these stage should be more descriptive in personal opinion

OR if the intervention is non-evasive, you can assume MDE of say 10 percentage point or whatever based on expert advice and what you anticipate the impact of the intervention would be and go through the rigorous power analyses calculation (type I error, power of the study, design effect etc). In that case, all these inferential models can be justified

See the following references.

Teresi, J. A., Yu, X., Stewart, A. L., & Hays, R. D. (2022). Guidelines for designing and evaluating feasibility pilot studies. Medical care, 60(1), 95-103.

Kraemer HC, Mintz J, Noda A, et al. Caution regarding the use of pilot studies to guide power calculations for study proposals. Arch Gen Psychiatry 2006;63(5):484–489.

There was no specific mention of how the data will be analyzed, which is a crucial aspect that the authors should kindly include. Descriptive statistics may be examined but the specific method should be mentioned based on measurement scale of the indicator. For example, the mean and standard deviation for continuous measures (or median and interquartile range for non-normally distributed outcomes) and the frequency and percentage for categorical measures can be calculated overall and by subgroups (intervention and control) without necessarily reporting inferential statistics (p-values, etc).

********** 

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Study Protocol for a Randomized Controlled Trial: Integrating the ‘Time-limited Trial’ in the Emergency Department

PONE-D-24-11345R1

Dear Dr. Hashimoto,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Dong Wook Chang

Academic Editor

PLOS ONE