PONE-D-21-01394

Using numerical modelling and simulation to assess the ethical burden in clinical trials and how it relates to the proportion of responders in a trial sample

PLOS ONE

Dear Dr. Boissel,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

ACADEMIC EDITOR: As per previous comments.

Please submit your revised manuscript by Jul 05 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Dylan A Mordaunt

Academic Editor

PLOS ONE

Additional Editor Comments:

Thank you for your submission. The reviewers have found some interest in this, however also some significant suggestions for consideration. Quantitation and modelling of ethical considerations has value insofar as the ability to operationalize ethical considerations in decision-making frameworks. I think there's value in revision and resubmission, albeit the suggestions are not insubstantial.

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. PLOS requires an ORCID iD for the corresponding author in Editorial Manager on papers submitted after December 6th, 2016. Please ensure that you have an ORCID iD and that it is validated in Editorial Manager. To do this, go to ‘Update my Information’ (in the upper left-hand corner of the main menu), and click on the Fetch/Validate link next to the ORCID field. This will take you to the ORCID site and allow you to create a new iD or authenticate a pre-existing iD in Editorial Manager. Please see the following video for instructions on linking an ORCID iD to your Editorial Manager account: https://www.youtube.com/watch?v=_xcclfuvtxQ

3. Thank you for stating the following in the Funding Statement Section of your manuscript:

[This work was supported by RESSTORE project (www.resstore.eu) funded by the European Commission under the H2020 program (Grant Number 681044).Authors are independent of the funder.]

We note that you have provided funding information that is not currently declared in your Funding Statement. However, funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form.

Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows:

 [The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.]

Please include your amended statements within your cover letter; we will change the online submission form on your behalf.

4. We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially identifying or sensitive patient information) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. Please see http://www.bmj.com/content/340/bmj.c181.long for guidelines on how to de-identify and prepare clinical data for publication. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: PONE-D-21-01394

Using numerical modelling and simulation to assess the ethical burden in clinical trials and how it relates to the proportion of responders in a trial sample

In the present analysis the authors analyse the number of participants within a trial who failed to have an event (death) within a 2-armed parallel RCT. They argue that a proportion of participants fail to benefit from participation on the basis that they never would have had the event in question, and that removing these individuals from trials, such that only responders are included would minimise ethical losses. The authors illustrate this with data from the Empa-REG-Outcome trial.

While this is perhaps a nice example as a statistical exercise, the challenge is what to do with the paper. The takeaway – we should try and have trials that are 100% responders – does not need the mathematical explanation. If we knew that, we wouldn’t need a trial we could just develop a drug and then give it the people we knew would respond.

Instead, the reality is that we do not know who will respond, and if the drug is given to a patient population (note this is the frame upon which the trial must be based) then there will likely be some group – based on some identified characteristic – that who benefits and some that does not.

I also have some specific comments:

1.The concept of ethical losses is not fleshed out; the manuscript would benefit from a clearer description of this concept. This is important given that later (p9lines 179,182) the authors talk about different types of ethical loss.

2.The authors state that “Despite their wide acceptance, questions have been raised about the ethics of randomizing participants to conventional treatments when deemed less effective, to placebo, or to an inadequate treatment if the therapy tested turns out to be less effective or potentially harmful.” – each of these have a specific context that the current phrasing excludes. For example, the initial component of the sentence likely relates to disruption of equipoise about the treatments; concerns about randomisation to placebo largely attend when treatment is available and so reflects the withholding of know treatment, and the third appears to be a post hoc determination.

3.The line about a different setting seems somewhat irrelevant to the manuscript.

4.I would disagree that consent is seen as a global answer to the above issues; but rather is a necessary component to conveying the state of knowledge. I don’t think anyone would suggest that a placebo becomes acceptable on the basis of consent. The acceptability of the placebo is an independent consideration.

5.On p3, lines 64 – 70, the authors write: “It is known that when a new treatment is being trialed, some patients will present the event the treatment is supposed to prevent as opposed to the “responders” who will show the expected reaction to treatment. Identification of responders is a timely and challenging issue that remains poorly studied, mainly because it is difficult to predict who responders and non-responders will be. But beyond the importance of identifying responders in the era of personalized medicine, the consequences of enrolling non-responders in RCTs remain unexplored.” – If this is the motivation, the discussion doesn’t reflect this. What is the importance with respect to personalised medicine (would the identification not be post-hoc analysis to try and find some causally related characteristic?)

6.On lines 72-78 the authors write “More precisely, the objective of our work is to show how can RCT simulations be used to address the consequences on the researcher’s ethical obligation of diluting the responders in the trial sample. Our simulations do not aim at quantitatively defining adequate parameters of an RCT but intend to draw upon the qualitative perception of the issue of ethical loss linked to the current way of RCTs are planned, especially when it comes to eligibility criteria. To that extent, our approach focuses on treatment efficacy.”

a.What is the ethical obligation of diluting the responders in the trial sample? I am not clear what this is referring to.

b.I am not sure how the work addresses the consequences of the analysis.

Methods

Given the extensive exposition of the formulas, why were they not used? It seems strange to spend the time detailing the approach, only to then not use it and use an approach that isn’t described.

The terminology changes such that P is used for placebo, then the subscript c (for control?) is subsequently used.

The lumping together of the group d is confusing. It would have been more intuitive to have these designated as two groups d1 and d2 to more explicitly note that d1 are those alive who would never have had the event, and those d2 those that would have had the event but benefited from the treatment.

On p8, lines 157-162, the authors state: “Looking at the figures in Table 1 from a purely arithmetic point of view, the gold standard statistical significance can be achieved, among other means, by decreasing ‘c’ and ‘d’, the number of recruited subjects who will not present the event, whatever the arm they have been randomly allocated to. These ‘c + d’ participants do not carry any information regarding the primary objective of the trial.” – this could do with greater exposition abut how this is achieved, at present this is left unclear.

Can the authors explain more why mortality trumps all other benefits such that they can be disregarded – this claim surely needs to be better justified? In terminal or palliative care, for example, a drug which may not prolong life or stave off death may be beneficial if it sufficiently advances quality of life (one may contend).

Indeed, the simplicity of the analysis does not reflect reality and the other outcomes of interest (indeed in the authors own trial the outcome was a composite outcome indicating that there was at least some suggestion that other outcomes may be as important.

Doesn’t the assumption about probability of death mask the complexity of identifying non-responders and responders? Again, this goes to the utility of the analysis – if the assumptions are not valid, how should the results be taken?

Reviewer #2: Summary: This manuscript reports on a complex hypothetical study which uses statistical and numerical modelling in an attempt to map out the proportion of ‘responders’ and ‘non responders’ in randomized control trials (RCTs) for comparison of new drugs or treatments, in an attempt to reduce the number of ‘non-informative participants’ or ‘unnecessary’ participants in RCTs. By identifying such non-responders in RCTs, the authors argue that this would help to minimize ‘ethical losses’ characterized for example as “exposure to side-effects, loss of access to standard care, or loss of access to 182 the best available and tailored treatment.”

While he manuscript is original ad attempts to resolve some of the ethical and moral dilemmas associated with the RCT format for investigational drugs or therapy. The manuscript as currently written is not easily comprehensible to the ‘non-expert’ or general reader and will require some modifications to enhance clarity and reduce some identified ambiguities in the current manuscript prior to acceptance for publication.

Suggested areas for further clarification and improvement of the manuscript:

1.a). In the Abstract the authors appear to use the following terms interchangeably i.e., “non-responders,”; “unnecessary participants” and “non-informative participants” …. The question that comes to mind is this…. Do these terms refer to the same fragment of the sample population, or are they different in anyway?

b). Again, on Page 8, Lines 176-177, the authors state “non-responders, non-informative and unnecessary participants, totaling (sp) the number of non-responders and non-informative participants.”

c). Furthermore, on Page 8, Lines 168-171, the authors state “Elaborating on this same reasoning, the non-responders are unnecessary participants for they do not bring into the trial a part of the experimental treatment efficacy and they cannot (when they were allocated to the control arm) or do not (experimental treatment arm) benefit from the treatment.”

One would like to suggest that this apparent use of these different terminologies of ‘non-responder’, ‘unnecessary participants’ and ‘non-informative participants’ creates a confusion for the reader who has to try to decipher the meaning of each of this categories…I therefore suggest that the authors should clearly define each of these categories…Are they the same or are they different?...If they are the same then say so…if they are different , then clearly define how they are different from each other? Are some of these categories essential to the RCT format…if so, which ones are essential or non-essential?... The way to approach this is suggested in 2. Below:

2.On page 5, Line 104…the authors have inserted a subheading for “Responders and Non-Responders”. However, after including a section “Definition of Responders” and clearly elaborating on “Responders”, the reader is left in the dark about a definition for “Non-Responders”? Therefore, following on from the questions raised in Paragraphs 1a, b, c above…One would like to suggest that the authors should clearly define who the ‘Non-responders’ in this study are, and also to indicate whether they are the same as ‘unnecessary participants’ and ‘non-informative participants…. This will assist in enhancing the clarity and comprehension of the arguments raised in the rest of the article/manuscript….

3.On page 9 Lines 178-182, the authors have added a subsection on “Ethical losses and unnecessary participants”. Here the authors state, “A source of recurring dispute within RCTs is their ethical acceptability. For an individual, participating in an RCT can mean different types of ethical loss, for example, exposure to side-effects, loss of access to standard care, or loss of access to the best available and tailored treatment.”. One would like o suggest that other important potential ethical losses which ought to be discussed would be issues of ‘moral distress and justice’…For example where individuals have volunteered for an RCT with the hope of gaining a potential benefit or even contributing to science only to find out that there their participation was perchance ‘unnecessary’…This can create some form of ‘moral distress’ and a reluctance to participate in future RCTs. Furthermore, with regards to justice, where individuals have invested time and effort or money e.g., taking time off work to participate in an RCT, it could be considered an injustice or unjust and unethical to use peoples time and effort without any justifiable benefits either for themselves or to society in general…. This would be the case regardless of whether the individual subjects participation was covered under the informed consent doctrine? …

4.On pages 17-18, Lines 333-345, The authors have described the concept of ‘equipoise in RCTs’. One would like to suggest that moving this section to the ‘Introduction’ section from the ‘Discussion’ which could serve as a background to RCT studies and provide a justification on why it would be important to identify “responders” and “non-responders” in RCTs….??

5.Insert a statement of limitations which identifies potential limitations to assumptions made in this study…For example are there any other confounding variables which could impact on the assumptions made for this study? E, g. What about participants who withdraw prematurely from an RCT... How will this impact on the percentage or proportion of ‘responders’ and ‘non-responders’…. Because the assumptions illustrated in this case are based on the principle of ‘all things being equal’…??

6.Finally, the statistical parameters and calculations for this study should be further reviewed by another qualified biostatistician

Minor Corrections: Few typographical and grammatical errors…

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Sylvester C. Chima, MD, LLM, LLD

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Attachments

Attachment

Submitted filename: Manuscript Review Report- PONE-D-21-01394- May-2021.docx

Using numerical modeling and simulation to assess the ethical burden in clinical trials and how it relates to the proportion of responders in a trial sample

PONE-D-21-01394R1

Bonjour Professor Boissel,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Merci beaucoup,

Dylan A Mordaunt

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #2: All comments have been addressed

Reviewer #3: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #2: Yes

Reviewer #3: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2: Yes

Reviewer #3: I Don't Know

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #2: Yes

Reviewer #3: No

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #2: (No Response)

Reviewer #3: This paper uses modeling to demonstrate the effect of increasing the number of responders on ‘ethical loss,’ i.e., the number of individuals for whom participating in research is ‘unnecessary,’ as they are either non-responders or because their participation is non-informative. Intuitively, they find that as the number of responders in a trial increases, the number of unnecessary participants decreases (as does sample size) and trial power increases.

As these results are intuitive, I am struggling to see what this analysis adds and encourage the authors to focus their time and attention on the ‘so what’ of this paper. We know non-responders are a problem for RCTs. We also know that it is not possible to identify non-responders a priori. What then is the authors’ takeaway message?

I have made a number of suggestions for further clarification and improvement of the manuscript:

1. Line 50 I suggest replacing the phrase “loss of chance” with “loss of opportunity” throughout the manuscript.

2. Line 49-54 I am unclear about the link that is being drawn between the concept of ‘ethical loss’ and equipoise and suggest that reorganization of the introduction would be helpful. Namely, I suggest leading with a discussion of why RCTs are ethically fraught, followed by the issue of responders vs. non-responders and their effect on trials, and finally I would introduce the concept of ‘ethical loss’ and what the paper is trying to do.

Along those lines, I think it would be important to provide more substantive material on how non-responders dilute trials and the statistical difficulties such dilution produces. As it is currently written, I don’t think there is enough background information for a general reader to easily understand the material.

3. Line 116-118 The authors state that they will provide two examples of why it is important to separate the qualitative objectives of RCTs from the quantitative objectives and then only provide one.

4. Line 169 I recommend moving these definitions to earlier in the methods section. Perhaps you could rename the section entitled “Definition of a responder” to “Definitions” and then have sub-headings that distinguish between the different components of ‘ethical loss.’ Moreover, I found myself having to go back to the definitions several times as I was reading and encourage the authors to put together a definitions table for quick reference (if space permits).

5. Line 220-221 I would argue that whether a trial is ethical or not is entirely dependent on the trial’s risk-benefit analysis, not whether there is a positive benefit-loss for each participant (as long as risks to those participants are minimized). What’s more, the authors focus entirely on the direct benefits that a research participant might expect when participating in a trial (namely, getting access to a drug that works), however there are indirect benefits that are equally important, including participation in the production of generalizable knowledge. If the thrust of this paper is about the ethical impact of including ‘unnecessary participants’ in RCTs, I think it is critical to be clear about all of the relevant ethical considerations.

6. Line 212 In general, the manuscript tends to blur trial-level and participant-level risks and benefits, which is most apparent in the section entitled “Ethical issues and unnecessary participants.” Going back to my last point, if this is an ethics paper at heart, I encourage being very careful about the ethical considerations.

7. Line 331 The discussion section need significant work to highlight the meaning, importance and relevance of the results. In particular, the implications of the modeling need to be emphasized - why do the results matter?

The limitations section should follow the discussion.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: Yes: Sylvester C. Chima, MD, LL.M, LLD

Reviewer #3: No