Citation: Dal-Ré R, Moher D, Gluud C, Treweek S, Demotes-Mainard J, Carné X (2011) Disclosure of Investigators' Recruitment Performance in Multicenter Clinical Trials: A Further Step for Research Transparency. PLoS Med 8(12): e1001149. https://doi.org/10.1371/journal.pmed.1001149
Published: December 27, 2011
Copyright: © 2011 Dal-Ré et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: The authors received no specific funding for this article. Dr. D Moher is funded, in part, by a University Research Chair (University of Ottawa, Canada).
Competing interests: RDR reports being a GlaxoSmithKline employee (as Medical Director in Spain) until 2009 and owning stock in this firm. XC is the chairperson of the Spanish Commmittee of Drug Evaluation, an external advisory board to the Spanish Medicines Agency (AEMPS). DM is a member of the PLoS Medicine Editorial Board. The ideas and opinions expressed here are those of the authors and do not necessarily reflect those of the institutions or organizations they work for. All other authors reported having no competing interests.
Abbreviations: CT, clinical trial; MCT, multicenter clinical trial; REC, research ethics committee
Provenance: Not commissioned; externally peer reviewed.
- Many clinical trials are terminated before reaching the sample size needed to test the trials' hypotheses owing to poor recruitment.
- Registries, such as ClinicalTrials.gov, provide information on the main features of a multicenter clinical trial (MCT) to the general public.
- Site investigators are key to the success of MCTs; however, information on their recruitment performance is not publicly available.
- We propose that sponsors should disclose the recruitment targets of all site investigators on ClinicalTrials.gov before a trial starts as well as their final recruitment. Information on issues that could have affected recruitment should also be provided.
- This information will be of interest to different stakeholders such as patient organizations, sponsors, and MCT networks.
- Disclosing all site investigators' recruitment figures could prompt queries to the sponsor from the scientific community about regional subgroup analyses, to assess if ethnic or standard-of-care differences have an impact on treatment outcomes.
Transparency: A Fundamental Social Obligation for Clinical Research
After 60 years devoted to enhancing the methodology and ethics in clinical research, the last decade has been crucial to the scientific community in refining the transparency on conducting clinical trials (CTs), from their inception to the publication of results. A myriad of articles have been published on the design, conduct, conflicts of interest, reporting, and publication of CTs. Now a responsible investigator (or sponsor) involved in the design and conduct of a CT must, in addition to obtaining research ethics committee (REC) and, in many countries, regulatory approvals, register the protocol in a publicly accessible registry before recruiting the first participant and publish (or otherwise make publicly available) the results obtained –. There are many CT registries, 14 of which are included on the WHO International Clinical Trials Registry Platform . ClinicalTrials.gov , the largest with more than 90,000 CTs from around the world, provides a substantial amount of information to anybody interested in a trial. However, an important piece of information is lacking: recruitment information for each site participating in the CT. We believe in the benefits of increased transparency in all phases of clinical research, since it has been proven not only to build trust, but also to be one of the best tools to reduce bias, to improve credibility of the results, and to foster the efficient management of trials.
Recruitment: A Crucial Aspect in Conducting a Clinical Trial
A critical aspect to carefully consider in the planning of a CT is the number of participants to be recruited in a specified time period and to outline contingency plans if problems in recruitment arise. It is well known that recruiting participants can be a significant burden and is frequently the most difficult task in conducting a CT. Failure to reach the planned sample size within the agreed time frame and funding is common. Poor recruitment is a waste of resources and a potential abuse of participant goodwill. In the UK, only 31% of CTs sponsored by the Medical Research Council and the Health Technology Assessment Programme achieved their original recruitment target; furthermore, 55% did not reach the revised target . In 333 concluded UK public and charity-sponsored cancer trials that were started between 1971 and 2000, only 48% reached the planned sample size, whereas 20% of the CTs recruited less than 25% of the planned sample size . Recent US data showed that among 180 National Cancer Institute (NCI) Cancer Evaluation Program-sponsored CTs, activated between 2000 to 2004 and closed to accrual, 36% and 62% of phase 2 and 3 trials, respectively, did not attain their recruitment goals .
This pattern in publicly funded trials is also occurring in industry-sponsored trials. For example, in an industry-sponsored multicenter clinical trial (MCT) on asthma conducted in 11 countries, only 35% of sites succeeded (or exceeded) in recruiting the committed number of patients . In fact, some companies assume in the planning phase that up to 25% of the sites in a MCT will never enroll a single patient , which emphasizes the difficulty of recruiting participants. Reports from RECs also found a significant percentage of CTs with poor recruitment –. Overestimating the pool of eligible patients or the recruitment rate (known as “Lasagna's law”) occurs even though Good Clinical Practice guidelines state that investigators “should be able to demonstrate (e.g., based on retrospective data) a potential for recruiting the required number of suitable subjects within the agreed recruitment period,” and that “the investigator should have sufficient time to properly conduct and complete the trial within the agreed trial period” . Recently a number of reviews on interventions to improve CT recruitment have been published ,.
Except for some early proof-of-concept CTs, a sample size estimation should be conducted and reported in the trial protocol for all randomized CTs—something that is frequently lacking ,—or a rationale for not doing so should be included in the study protocol. Data show that in up to 66%  to 75%  of CT reports, there is no reporting of the sample size estimation.
In planning a CT, sample size estimation will trigger many decisions about the number of recruitment sites (and even countries) required to successfully accrue participants. If the number of participants is such that the research question cannot be properly answered, the trial may risk becoming scientifically useless and ethically unacceptable: the CT will bring burdens and risks (e.g., adverse effects) to participants, with no medical benefit for future patients because of its lack of scientific usefulness (e.g., no statistically significant difference detected between treatment groups). The question that follows is: what about the many CTs that never recruit the number of participants needed to appropriately test the underlying hypothesis? Is this not a (subtle) way of not fulfilling the investigator–participant contract inherent in a CT? If the results are not published in peer-reviewed journals, the investigator or the sponsor should make the results available through a private registry (e.g., company website) and provide a link to the website through a public registry (e.g., ClinicalTrials.gov). If the obligation to making the results publicly available is fulfilled, a CT that has not recruited the number of participants needed is underpowered and less informative from the perspective of the individual CT; nonetheless, the data, may be of scientific use if they are used to conduct meta-analyses.
The Proposal: To Provide Information on All Site Principal Investigators' Recruitment Performance
Recruitment performance of clinical centers and investigators varies significantly , in both public- and private-sponsored CTs, although most drug MCTs are sponsored by industry . Many investigators will participate in a number of trials in their professional careers, sometimes even simultaneously for the same condition. Through public registries, such as ClinicalTrials.gov, access to information is made available on the scientific rationale, methods, sponsorship, scientific leadership, calendar, contacts, and locations. Although currently available CT registration enhances transparency, deficiencies have been found that undermine the full potential usefulness of registries . We believe that recruiting sites' principal investigators should be accountable for the conduct and outcome of their trial. This should be regarded similarly to the current practice of publishing MCT results in a manuscript with all recruiting researchers included as members of the “study group.” Hence, we propose not only to disclose the names, contact details, and affiliations of all sites' recruiting investigators in the registry before trial start, something that is currently being provided by a variety of sponsors , but also to reveal their recruitment commitment and the number of recruited participants (Table 1).
It should be highlighted that some sponsors do currently provide information on the recruitment status of all sites involved in an MCT. Thus, in ClinicalTrials.gov , near the site principal investigator contact details, the recruitment status is sometimes provided as “not yet recruiting,” “active, but not recruiting,” “recruiting,” “completed,” or “withdrawn”. We propose for a given site, that if the trial has more than one recruiting principal investigator, then information for all recruiting investigators should be provided. This information will allow readers to assess the performance of each site principal investigator. This information may be of interest to the REC (or institutional review board) assessing the qualifications of a given researcher (and CT underway or committed in the same population) when reviewing a CT protocol , and to patient organizations, hospital and university managers, CT networks and collaborative groups, public and private sponsors, and health authorities. Thus, sponsors willing to start a CT may wish to recruit investigators with a track record of fulfilling their commitments, hospital or university managers may want data on the performance of their investigators in an activity that can impact their institutions' prestige, and patient organizations may want to know the investigators who actually recruit most participants in CTs in their area of interest. Furthermore, since it is well known that both ethnic and standard-of-care differences can impact treatment outcomes in different geographical regions –, having access to the percentage of participants coming from different countries (and, hence, regions) could prompt queries to the sponsor from experts about regional subgroup analyses. In addition, providing recruitment data could be of interest to researchers conducting prospective meta-analysis and individual patient data meta-analysis to examine specific outcomes and harms. Participants in a trial have the right to be informed on the results obtained : if an MCT is not concluded because of recruitment issues, recruiting site principal investigators could transparently inform trial participants wishing to know which centers did not deliver as expected, because the data are available on the registry. Finally, misconduct and fraud in clinical research does happen ,. Full transparency regarding recruitment goals and results should help to reduce the risk of misconduct and fraud.
How to Comply with the Proposal: The Practicalities
Registering investigators' recruitment performance information in MCTs should be straightforward, since a record of all research sites is always regularly up-dated (on a real-time enrollment basis in many trials) by the sponsor or the coordinating center. Hence, this proposal represents a modest burden on sponsors. It is reasonable to believe that the fact of having these data publicly disclosed will encourage site recruiting investigators to ask the sponsor to update the information; at the end of the day, each investigator should take responsibility for his/her own reputation in relation to patients, institution, and other researchers. At the very least, the recruitment final data per site principal investigator should be provided when recruitment of participants has been concluded (having achieved or not achieved the trial's target number of participants). In any case, information regarding recruitment per site is usually included in the MCT final report and when reporting industry-sponsored trials results to regulatory agencies .
Importantly, once the initial list of recruiting centers and site principal investigators (and, following our proposal, their recruitment targets) is provided, the list will not usually change unless (a) an investigator is asked to stop his/her contribution to the trial because of a lack of recruitment; in this case, the sponsor should include the appropriate information on the registry, or (b) new recruiting centers or investigators are added to the original list, most likely as a contingency plan to enhance the trial's poor recruiting rate to date. After the initial target between the sponsor and each site investigator is agreed upon, further agreements should be registered; a record of these agreements will be available on the registry's archive site thanks to the audit trail .
Any related issue that could affect recruitment in a site should be disclosed, thus providing the reader with the necessary information to understand what happened during the recruitment period of the trial. Among the non-site investigator related issues, and to facilitate submission of information, we propose that this information should be classified in a number of “closed” reasons, as shown in Table 1. Similarly, any unexpected investigator-related reason that could impact his/her recruitment performance should be recorded. We suggest only a “closed” reason (“health issues”) in this section.
Since all parties involved (from research-funding agencies and regulatory agencies to journal editors and industry) are asking for transparency in CTs ,, we propose that its level should be taken beyond the current limits. We acknowledge that several critical issues need to be addressed in the near future to enhance CT transparency. Among these are the urgent need for sponsors to provide complete and accurate information as required by public registries  as well as access to trial protocols and raw databases . In this respect, we propose that all MCTs registered in publicly available registries, such as ClinicalTrials.gov , should provide detailed information about the names, contact details, recruitment commitments, and recruitment deliveries of all site principal investigators involved. This information will make them accountable for the MCT performance not only to the sponsor, but also to all participating patients or healthy volunteers and recruiting investigators, all other interested researchers in the field, and, ultimately, society as a whole. In fact, disclosing the recruitment performance of all site recruiting principal investigators is a small but relevant part of an MCT's raw data; they must be considered as belonging to the scientific study information that should be openly shared, something that is currently being promoted by private and public funding organizations . Similar to other initiatives established to improve transparency, such as the CONSORT statement—although this was enforced by journal editors—moves by sponsors to comply with this proposal are likely to be gradual ,. We view reporting of site-level recruitment to be a logical next step in improving trial transparency, and we hope that many sponsors will help in implementing this initiative.
Translation of the summary points into Spanish, by RD-R.
Wrote the first draft of the manuscript: RDR. Contributed to the writing of the manuscript: RDR DM CG ST JDM XC. ICMJE criteria for authorship read and met: RDR DM CG ST JDM XC. Agree with manuscript results and conclusions: RDR DM CG ST JDM XC. Involved in the critical revision of the manuscript for important intellectual content: RDR DM CG ST JDM XC.
- 1. De Angelis C, Drazen JM, Frizelle FA, Haug C, Hoey J, et al. (2004) Clinical trial registration: a statement from the international Committee of Medical Journal Editors. Ann Intern Med 141: 477–478.
- 2. Sim I, Chan AW, Gulmezoglu AM, Evans T, Pang T (2006) Clinical trial registration: transparency is the watchword. Lancet 367: 1631–1633.
- 3. World Medical Association (2008) Declaration of Helsinki. Available: http://www.wma.net/en/10home/index.html. Accessed 17 July 2011.
- 4. Food and Drug Administration Amendments Act of 2007. Available: http://frwebgate.access.gpo.gov/cgi-bin/getdoc.cgi?dbname=110_cong_public_laws&docid=f:publ085.110. Accessed 17 July 2011.
- 5. World Health Organization (2011) International Clinical Trials Registry Platform (ICTRP). Available: http://www.who.int/ictrp/en/. Accessed 13 October 2011.
- 6. National Institutes of Health (2011) ClinicalTrials.gov. Available: http://clinicaltrials.gov/. Accessed 22 May 2011.
- 7. McDonald AM, Knight RC, Campbell MK, Entwistle VA, Grant AM, et al. (2006) What influences recruitment to randomised controlled trials? A review of trials funded by two UK funding agencies. Trials 7: 9.
- 8. Vale C, Stewart L, Tierney J, the UK Coordinating Committee for Cancer Research. National Register of Cancer (2005) Trends in UK cancer trials: results from the UK Coordinating Committee for Cancer Research National Register of Cancer Trials. Br J Cancer 92: 811–814.
- 9. Cheng SK, Dietrich MS, Dilts M (2010) A sense of urgency: evaluating the link between clinical trial development time and the accrual performance of cancer therapy evaluation program (NCI-CTEP) sponsored studies. Clin Cancer Res 16: 5557–5563.
- 10. Dal-Ré R, Ortega R, Espada J (1998) Eficiencia de los investigadores en la selección de los enfermos para ensayos clínicos: a propósito de un estudio multinacional. [Efficiency of investigators in recruitment of patients for clinical trials: a propos of a multinational study] Med Clin (Barc) 110: 521–523.
- 11. Malakoff D (2008) Clinical trials and tribulations. Spiraling costs threaten gridlock. Science 322: 210–213.
- 12. Pich J, Carné X, Arnaíz JA, Gómez B, Trilla A, et al. (2003) Role of a research ethics committee in follow-up and publication of results. Lancet 361: 1015–1016.
- 13. Wise P, Drury M (1996) Pharmaceutical trials in general practice: the first 100 protocols. An audit by the clinical research ethics committee of the Royal College of General Practitioners. BMJ 313: 1245–1248.
- 14. Decullier E, Lheritier V, Chapuis F (2005) Fate of biomedical research protocols and publication bias in France: retrospective cohort study. BMJ 331: 19. doi:10.1136/bmj.38488.385995.8F.
- 15. International Conference on Harmonisation of technical requirements for Registration of Pharmaceuticals for Human Use (2006) ICH Harmonised Tripartite Guideline. Guideline for Good Clinical Practice. E6 (R1) June 1996. Available: http://www.ich.org/home.html. Accessed 17 July 2011.
- 16. Caldwell PHY, Hamilton S, Tan A, Craig JC (2010) Strategies for increasing recruitment to randomized controlled trials: systematic review. PLoS Med 7: e1000368.
- 17. Treweek S, Mitchell E, Pitkethly M, Cook J, Kjeldstrøm M, et al. (2010) Strategies to improve recruitment to randomised controlled trials. Cochrane Database Syst Rev MR000013.
- 18. Moher D, Dulberg CS, Wells GA (1994) Statistical power, sample size and their reporting in randomized controlled trials. JAMA 272: 122–124.
- 19. Chan AW, Hróbjartsson A, Jørgensen KJ, Gøtzsche PC, Altman DG (2008) Discrepancies in sample size calculations and data analyses reported in randomised trials: comparison of publications with protocols. BMJ 337: a2299.
- 20. Gluud C (2006) The culture of designing hepato-biliary reandomised trials. J Hepatol 44: 607–615.
- 21. Bourgeois FT, Murthy S, Mandl KD (2010) Outcome reporting among drug trials registered in ClinicalTrials.gov. Ann Intern Med 153: 158–166.
- 22. Zarin DA, Tse T, Williams R, Califf RM, Ide NC (2011) The ClinicalTrials.gov results database—update and key issues. N Engl J Med 364: 852–860.
- 23. Blair JEA, Zannad F, Konstam MA, Cook T, Traver B, et al. (2008) Continental differences in clinical characteristics, management, and outcomes in patients hospitalized with worsening heart failure. JACC 52: 1640–1648.
- 24. Ali M, Atula S, Bath PM, Grotta J, Hacke W, et al. (2009) Stroke outcome in clinical trials patients deriving from different countries. Stroke 40: 35–40.
- 25. Wallentin L, Becker RC, Budaj A, Cannon CP, Emanuelsson H, et al. (2009) Ticagrelor versus clopidogrel in patients with acute coronary syndromes. N Engl J Med 361: 1045–1057.
- 26. Kalb PE, Koehler KG (2002) Legal issues in scientific research. JAMA 287: 85–91.
- 27. Steen RG (2011) Retractions in the scientific literature: is the incidence of research fraud increasing? J Med Ethics 37: 249–253.
- 28. European Medicines Agency (2010) Clinical trials submitted in marketing authorization applications to the EMA. Overview of patient recruitment and geographical location of investigator sites. EMA/INS/GCP/154352/2010. 5 November 2010. Available: http://www.ema.europa.eu/docs/en_GB/document_library/Other/2009/12/WC500016819.pdf. Accessed 17 July 2011.
- 29. Boulton G, Rawlins M, Vallance P, Walport M (2011) Science as public enterprise: the case for open data. Lancet 377: 1633–1635.
- 30. Chan AW (2011) Access to clinical trial data. BMJ 342: 117.
- 31. Walport M, Brest P (2011) Sharing research data to improve public health. Lancet 377: 537–539.
- 32. Hopewell S, Altman DG, Moher D, Schulz KF (2008) Endorsement of the CONSORT Statement by high impact factor medical journals: a survey of editors and journal ‘instructions to authors’. Trials 9: 20.
- 33. Treweek S, Loudon K (2011) Incomplete reporting of recruitment information in breast cancer trials published between 2003 and 2008. J Clin Epidemiol 64: 1216–1222.