Dear Dr Ineichen,

Thank you for submitting your manuscript entitled "From mice to men: Evaluating the success rate of biomedical therapies' journey from animals to humans – an umbrella review" for consideration as a Meta-Research Article by PLOS Biology. Please accept my apologies for the delay incurred during our annual team retreat.

Your manuscript has now been evaluated by the PLOS Biology editorial staff, as well as by an academic editor with relevant expertise, and I'm writing to let you know that we would like to send your submission out for external peer review.

However, before we can send your manuscript to reviewers, we need you to complete your submission by providing the metadata that is required for full assessment. To this end, please login to Editorial Manager where you will find the paper in the 'Submissions Needing Revisions' folder on your homepage. Please click 'Revise Submission' from the Action Links and complete all additional questions in the submission questionnaire.

Once your full submission is complete, your paper will undergo a series of checks in preparation for peer review. After your manuscript has passed the checks it will be sent out for review. To provide the metadata for your submission, please Login to Editorial Manager (https://www.editorialmanager.com/pbiology) within two working days, i.e. by Dec 13 2023 11:59PM.

If your manuscript has been previously peer-reviewed at another journal, PLOS Biology is willing to work with those reviews in order to avoid re-starting the process. Submission of the previous reviews is entirely optional and our ability to use them effectively will depend on the willingness of the previous journal to confirm the content of the reports and share the reviewer identities. Please note that we reserve the right to invite additional reviewers if we consider that additional/independent reviewers are needed, although we aim to avoid this as far as possible. In our experience, working with previous reviews does save time.

If you would like us to consider previous reviewer reports, please edit your cover letter to let us know and include the name of the journal where the work was previously considered and the manuscript ID it was given. In addition, please upload a response to the reviews as a 'Prior Peer Review' file type, which should include the reports in full and a point-by-point reply detailing how you have or plan to address the reviewers' concerns.

During the process of completing your manuscript submission, you will be invited to opt-in to posting your pre-review manuscript as a bioRxiv preprint. Visit http://journals.plos.org/plosbiology/s/preprints for full details. If you consent to posting your current manuscript as a preprint, please upload a single Preprint PDF.

Feel free to email us at plosbiology@plos.org if you have any queries relating to your submission.

Kind regards,

Roli Roberts

Roland Roberts, PhD

Senior Editor

PLOS Biology

rroberts@plos.org

Dear Dr Ineichen,

Thank you for your patience while your manuscript "From mice to men: Evaluating the success rate of biomedical therapies' journey from animals to humans – an umbrella review" went through peer-review at PLOS Biology. Your manuscript has now been evaluated by the PLOS Biology editors, an Academic Editor with relevant expertise, and by three independent reviewers.

You'll see that reviewer #1 is very positive and has a list of minor textual and presentational requests. Reviewer #2 is also positive and has no requests. Reviewer #3 is also favourable, and has a number of requests, most of which relate to clarity around the methodology. However, while we appreciate that this reviewer's final request, which is to update the study before resubmitting, will further improve the paper, we will leave it to your discretion.

IMPORTANT: please also attend to the following:

a) Please comply with PLOS' Data Policy (https://journals.plos.org/plosbiology/s/data-availability); specifically, we need you to supply the numerical values underlying Figs 2. 4ABC, 5, S2ABC, S3, S4, S5, S6, S7, S8, either as a supplementary data file or as a permanent DOI’d deposition. I note that your study is already preregistered with OSF (https://osf.io/jh2d8), so perhaps you could deposit it there?

b) Please cite the location of the data clearly in all relevant main and supplementary Figure legends, e.g. “The data underlying this Figure can be found in S1 Data” or “The data underlying this Figure can be found in https://doi.org/10.5281/zenodo.XXXXX”

c) Please make any custom code available, either as a supplementary file or as part of your data deposition.

d) Your Data Availability Statement currently says “All data and code that support the findings of this study are available from the corresponding author, BVI, upon reasonable request, at benjamin.ineichen@uzh.ch” - this is not compliant with PLOS Data Policy, so please comply and update (see above).

e) Your Abstract is currently structured; please change it to an unstructured format (https://journals.plos.org/plosbiology/s/submission-guidelines#loc-abstract).

f) Please remove the beginning and end of your Title, to make it simply "Evaluating the success rate of biomedical therapies' journey from animals to humans"

In light of the reviews, which you will find at the end of this email, we are pleased to offer you the opportunity to address the comments from the reviewers in a revision that we anticipate should not take you very long. We will then assess your revised manuscript and your response to the reviewers' comments with our Academic Editor aiming to avoid further rounds of peer-review, although might need to consult with the reviewers, depending on the nature of the revisions.

We expect to receive your revised manuscript within 1 month. Please email us (plosbiology@plos.org) if you have any questions or concerns, or would like to request an extension.

At this stage, your manuscript remains formally under active consideration at our journal; please notify us by email if you do not intend to submit a revision so that we withdraw the manuscript.

**IMPORTANT - SUBMITTING YOUR REVISION**

Your revisions should address the specific points made by each reviewer. Please submit the following files along with your revised manuscript:

1. A 'Response to Reviewers' file - this should detail your responses to the editorial requests, present a point-by-point response to all of the reviewers' comments, and indicate the changes made to the manuscript.

*NOTE: In your point-by-point response to the reviewers, please provide the full context of each review. Do not selectively quote paragraphs or sentences to reply to. The entire set of reviewer comments should be present in full and each specific point should be responded to individually.

You should also cite any additional relevant literature that has been published since the original submission and mention any additional citations in your response.

2. In addition to a clean copy of the manuscript, please also upload a 'track-changes' version of your manuscript that specifies the edits made. This should be uploaded as a "Revised Article with Changes Highlighted " file type.

*Resubmission Checklist*

When you are ready to resubmit your revised manuscript, please refer to this resubmission checklist: https://plos.io/Biology_Checklist

To submit a revised version of your manuscript, please go to https://www.editorialmanager.com/pbiology/ and log in as an Author. Click the link labelled 'Submissions Needing Revision' where you will find your submission record.

Please make sure to read the following important policies and guidelines while preparing your revision:

*Published Peer Review*

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out. Please see here for more details:

https://blogs.plos.org/plos/2019/05/plos-journals-now-open-for-published-peer-review/

*PLOS Data Policy*

Please note that as a condition of publication PLOS' data policy (http://journals.plos.org/plosbiology/s/data-availability) requires that you make available all data used to draw the conclusions arrived at in your manuscript. If you have not already done so, you must include any data used in your manuscript either in appropriate repositories, within the body of the manuscript, or as supporting information (N.B. this includes any numerical values that were used to generate graphs, histograms etc.). For an example see here: http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1001908#s5

*Blot and Gel Data Policy*

We require the original, uncropped and minimally adjusted images supporting all blot and gel results reported in an article's figures or Supporting Information files. We will require these files before a manuscript can be accepted so please prepare them now, if you have not already uploaded them. Please carefully read our guidelines for how to prepare and upload this data: https://journals.plos.org/plosbiology/s/figures#loc-blot-and-gel-reporting-requirements

*Protocols deposition*

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

Thank you again for your submission to our journal. We hope that our editorial process has been constructive thus far, and we welcome your feedback at any time. Please don't hesitate to contact us if you have any questions or comments.

Sincerely,

Roli Roberts

Roland Roberts, PhD

Senior Editor

PLOS Biology

rroberts@plos.org

----------------------------------------------------------------

REVIEWERS' COMMENTS:

Reviewer #1:

[identifies himself as Jan-Bas Prins]

This manuscript describes a systematic review of published systematic reviews or scoping reviews with or without meta-analysis which investigate translation of interventions in animal models of human diseases. The authors name their approach an umbrella review. The study adds a dimension to the ongoing debate about the validity and translatability of pre-clinical testing in drug development for humans. Focussing on three phases in the trajectory of drug testing: the pre-clinical phase, the randomised clinical trial phase and approval by the competent authority, in this case the FDA. The authors used published search strategies and a purposely developed one to mine Medline, Embase, and Web of Science. They performed a risk of bias analysis in which they identified several studies with some concerns others with high risk of bias. They started with 3343 records for the first screening and ended with 122 studies for the qualitative synthesis and 62 for the quantitative synthesis/ meta-analysis. The study shows that although animal studies appear to translate better to humans than previously thought, there is still a very low rate of final approval. The manuscript is well-written and demonstrates sound research design, execution, and reporting.

I have only a few remarks and comments to make in order of appearance in the manuscript.

1. Material and methods:

a. Lines 74-76 - here it reads that Medline was searched with Ovid Medline interface. In other parts of the manuscript, reference is made to Pubmed, which is an interface to search Medline as well as additional biomedical content.

b. Line 79 - typo 'SYRLCE'

c. Line 81-84 - a 'broad' and a 'narrow' search strategy were tested for sensitivity before selecting one to the two. A threshold was set at 5% additional studies retrieved with the broad strategy. The authors 'did not'. It would be nice to know what the actual percentage was.

d. Line 96-97 - non-systematic reviews were excluded but retained for additional references. Only one additional article was found and added to the search results. This low number reflects the quality of the SRs included and could be discussed briefly.

e. Line 106-107 - add reference (or URL) for Universal Desktop Ruler.

f. Line 151 - add references (or URLs) to the packages survminer and survival respectively.

2. Results:

a. Lines 181-185 - It is unclear if studies that were labelled as at high risk of bias or had concerns raised were still included in the analysis and if so, what the potential impact is.

b. Lines 209-210 - the authors note 'that many animal studies were conducted subsequent to the publication of a corresponding clinical study'. Without the studies at hand, this can be interpreted in different ways. It would be helpful if the authors could elaborate on this notion by adding information to what extent the research questions of the animal studies were overlapping with or novel with respect to the clinical study.

c. Line 217 - (281 therapies, Figure 3) add A to figure 3.

d. Line 230 - reference to Supplementary figure 2 - in this figure the term 'survival time' is used. Could the alternative 'lag time' be considered?

e. Line 235-236 - what is the justification for the cut-off of less than 5 published animal studies?

f. Lines 236-239 - The numbers in Table 2 are different from those in the text. The legend of table 2 should include information on the cut-off number of published animal studies.

3. Discussion:

a. Lines 243-246 - although it may be considered as beyond the scope of the manuscript, the paper would gain from discussion on the differences between the therapies (and/ or models used) for neurological diseases and circulatory system diseases versus those for diseases with lower concordance. Also, in relation to the section of the discussion (lines 273-280) where the example of field of clinical neurology is presented to demonstrate the limited official approval of therapies while the results of animal studies and early clinical trials were promising.

b. Lines 291-292 - the fourth potential hurdle for successful translation is the quality of human studies i.e. underpowered and inappropriate outcome measures. These refer to serious flaws in experimental design and outcomes of assessment of protocols by competent authorities. Could the authors add examples or provide some level of quantification?

c. Lines 312-314 - I am not sure I understand this limitation since only one publication was added to those already identified by the SRs based on the excluded papers.

d. Lines 318-320 - Is this a limitation?

Reviewer #2:

[identifies himself as Jens Minnerup]

This is an excellent review article by leading experts in the field. The article will be of interest to a wide readership.

Congratulations to the authors.

I have no further comments.

Reviewer #3:

Thank you for the opportunity to review this very well-written paper evaluating the success rate of biomedical therapies' journey from animals to humans conducted as an umbrella review. This is a very important and timely review. The results are interesting and provide new evidence to inform effective and efficient translational pathways across multiple diseases.

Comments:

- Quality assessment: Can you please clarify the process for developing these questions and any piloting completed to ensure the clarity of interpretation. Were any inter-rater statistics completed? Are additional details for how to apply these questions able to provided in a supplemental to allow someone else to use in the future or reproduce your scores?

- It reads as though you found reviews and then did some linking of other studies based on interventions identified. Are you able to clarify these processes in text.

- Line 189: Qualitative summary. There is one subsection focused on translation in different biomedical fields. However, are there other subsections? If not, I would remove the subheading.

- Line 267: Consider the role of sample size and efficacy (ideal and controlled circumstances) vs effectiveness (real world scenarios). Many interventions have been found to show promise at early clinical trial phase, but the effect is diluted when conducted on a much larger (roe generalisable) sample.

- Can you comment on the leadership of trials ie investigator initiated or industry sponsored? Does this have any role to play in the quantitative outcomes examined?

- Limitations section: acknowledge this review is of English only studies

- Table 1: As well as the summary numbers, can you please list number of studies per unique disease, and include the number in any study, RCT and FDA per unique disease as well? I know the there is a lot of information in the supplemental, but it would be great if there is a way to bring any summary level (consider using advanced data visualisation options) to the main paper.

- Table 2: Positive/Negative/Neutral - please include information about this in the methods including which outcome was used to make this determination eg primary or any outcome.

- Line 96: add (3) in front of at full text …

- Line 190: consider breaking up the paragraphs for stroke, cardiac arrest and cancer for ease of reading. This may mean expanding a little for some areas.

- Recommend updating the search prior to re-submission to ensure recency, however, also appreciate the large volume of work that has gone into pulling this review together, and note the findings may not change given the nature of this review. I am happy to be guided by editorial preference and authorship team perspectives here.

I really enjoyed reading this paper and look forward to citing it in the future! Congratulations on comprehensively pulling this large body of evidence together.

Dear Dr Ineichen,

Thank you for your patience while we considered your revised manuscript "Evaluating the success rate of biomedical therapies' journey from animals to humans" for publication as a Research Article at PLOS Biology. This revised version of your manuscript has been evaluated by the PLOS Biology editors and the Academic Editor. Please accept my apologies for the delay incurred. I should clarify that we had extreme difficulty communicating with the previous Academic Editor, and we have therefore switched to a new Academic Editor who will handle the paper for the rest of its journey.

Based on our Academic Editor's assessment of your revision, we are likely to accept this manuscript for publication, provided you satisfactorily address the points raised by the Academic Editor and the other data and other policy-related requests.

IMPORTANT - please attend to the following:

a) Please address the concerns from the (new) Academic Editor. You can find their actual comments at the foot of this email. My understanding is that they want you to 1. make it clear in the manuscript exactly how the intervention effect was classified, and 2. if you used one of the more problematic methods (e.g. relying on author conclusion), then this should be identified clearly as a limitation, with some discussion of the potential issues (spin, etc.).

b) Please could you change your title to something that encapsulates some of the findings? We suggest something like "Evaluating the success rate of biomedical therapies from animals to humans shows a 5% regulatory approval rate after 10 years"

As you address these items, please take this last chance to review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the cover letter that accompanies your revised manuscript.

We expect to receive your revised manuscript within two weeks.

To submit your revision, please go to https://www.editorialmanager.com/pbiology/ and log in as an Author. Click the link labelled 'Submissions Needing Revision' to find your submission record. Your revised submission must include the following:

- a cover letter that should detail your responses to any editorial requests, if applicable, and whether changes have been made to the reference list

- a Response to Reviewers file that provides a detailed response to the reviewers' comments (if applicable, if not applicable please do not delete your existing 'Response to Reviewers' file.)

- a track-changes file indicating any changes that you have made to the manuscript.

NOTE: If Supporting Information files are included with your article, note that these are not copyedited and will be published as they are submitted. Please ensure that these files are legible and of high quality (at least 300 dpi) in an easily accessible file format. For this reason, please be aware that any references listed in an SI file will not be indexed. For more information, see our Supporting Information guidelines:

https://journals.plos.org/plosbiology/s/supporting-information

*Published Peer Review History*

Please note that you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out. Please see here for more details:

https://blogs.plos.org/plos/2019/05/plos-journals-now-open-for-published-peer-review/

*Press*

Should you, your institution's press office or the journal office choose to press release your paper, please ensure you have opted out of Early Article Posting on the submission form. We ask that you notify us as soon as possible if you or your institution is planning to press release the article.

*Protocols deposition*

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

Please do not hesitate to contact me should you have any questions.

Sincerely,

Roli Roberts

Roland Roberts, PhD

Senior Editor

rroberts@plos.org

PLOS Biology

------------------------------------------------------------------------

DATA NOT SHOWN?

- Please note that per journal policy, we do not allow the mention of "data not shown", "personal communication", "manuscript in preparation" or other references to data that is not publicly available or contained within this manuscript. Please either remove mention of these data or provide figures presenting the results and the data underlying the figure(s).

------------------------------------------------------------------------

COMMENTS FROM THE ACADEMIC EDITOR [lightly edited]:

I read the answers to reviewers and the last version of the manuscript. I agree that the authors adequately answers all comments and revised their paper accordingly.

I must nevertheless say that I do not feel very comfortable in their classification of each study as whether the intervention had a positive, negative, mixed, or neutral effect on the corresponding disease outcome.

It seems quite unclear to me how they could do this: which outcome they considered (they state the main study outcome but often its difficult to identify or there are many and we don't know which one was prespecified), did they focus on p-value, effect estimates, confidence interval, consistency with other outcome assessed or did they use the author conclusion, which raises the issue of spin. This is slightly highlighted by reviewer 3 considering a specific study.

This approach is also questionable as the review authors "observed a progressive reduction in efficacy from animal research to early clinical trials and then to phase 3 clinical trials. "

Otherwise the rest of the paper did not raise any issue from me.

Dear Dr Ineichen,

Thank you for the submission of your revised Meta-Research Article "Analysis of animal-to-human translation reveals that only 5% of animal-tested therapeutic interventions obtain regulatory approval for human applications" for publication in PLOS Biology. On behalf of my colleagues and the Academic Editor, Isabelle Boutron, I'm pleased to say that we can in principle accept your manuscript for publication, provided you address any remaining formatting and reporting issues. These will be detailed in an email you should receive within 2-3 business days from our colleagues in the journal operations team; no action is required from you until then. Please note that we will not be able to formally accept your manuscript and schedule it for publication until you have completed any requested changes.

IMPORTANT: You'll see that I've taken the liberty of doing a minor edit to your title (my Editor-in-Chief has an aversion to punctuation in titles).

Please take a minute to log into Editorial Manager at http://www.editorialmanager.com/pbiology/, click the "Update My Information" link at the top of the page, and update your user information to ensure an efficient production process.

PRESS: We frequently collaborate with press offices. If your institution or institutions have a press office, please notify them about your upcoming paper at this point, to enable them to help maximise its impact. If the press office is planning to promote your findings, we would be grateful if they could coordinate with biologypress@plos.org. If you have previously opted in to the early version process, we ask that you notify us immediately of any press plans so that we may opt out on your behalf.

We also ask that you take this opportunity to read our Embargo Policy regarding the discussion, promotion and media coverage of work that is yet to be published by PLOS. As your manuscript is not yet published, it is bound by the conditions of our Embargo Policy. Please be aware that this policy is in place both to ensure that any press coverage of your article is fully substantiated and to provide a direct link between such coverage and the published work. For full details of our Embargo Policy, please visit http://www.plos.org/about/media-inquiries/embargo-policy/.

Thank you again for choosing PLOS Biology for publication and supporting Open Access publishing. We look forward to publishing your study. 

Sincerely,

Roli Roberts

Roland G Roberts, PhD, PhD

Senior Editor

PLOS Biology

rroberts@plos.org