The Quality of Registration of Clinical Trials: Still a Problem

Introduction The benefits of clinical trials registration include improved transparency on clinical trials for healthcare workers and patients, increased accountability of trialists, the potential to address publication bias and selective reporting, and possibilities for research collaboration and prioritization. However, poor quality of information in registered records of trials has been found to undermine these benefits in the past. Trialists' increasing experience with trial registration and recent developments in registration systems may have positively affected data quality. This study was conducted to investigate whether the quality of registration has improved. Methods We repeated a study from 2009, using the same methods and the same research team. A random sample of 400 records of clinical trials that were registered between 01/01/2012 and 01/01/2013 was taken from the International Clinical Trials Registry Platform (ICTRP) and assessed for the quality of information on 1) contact details, 2) interventions and 3) primary outcomes. Results were compared to the equivalent assessments from our previous study. Results There was a small and not statistically significant increase from 81.0% to 85.5% in the percentage of records that provided a name of a contact person. There was a significant increase from 68.7% to 74.9% in the number of records that provided either an email address or a telephone number. There was a significant increase from 44.2% to 51.9% in the number of intervention arms that were complete in registering intervention specifics. There was a significant increase from 38.2% to 57.6% in the number of primary outcomes that were specific measures with a meaningful timeframe. Approximately half of all trials continued to be retrospectively registered. Discussion There have been small but significant improvements in the quality of registration since 2009. Important problems with quality remain and continue to constitute an impediment to the meaningful utilization of registered trial information.


Introduction
Clinical trials registration is now broadly considered an ethical and scientific responsibility. [1][2][3][4][5][6][7][8] In the past fifteen years, national and regional trial registries have been established in Africa, Asia, Australia/Oceania, Europe, North America and South America. [9] The WHO International Clinical Trials Registry Platform (ICTRP) was established in 2005 with the aim of bringing registered trial data from different trial registries together and creating a single point of access to information on all clinical trials conducted globally. [10] It now combines data from 15 national and regional clinical trial registries, offering access to data from more than 200,000 trials.
There are important advantages to the increased transparency on clinical trial conduct and reporting brought about by these developments. It improves access to information on clinical trials for healthcare workers, researchers and patients [11,12]; it allows for steps to be taken against publication bias and selective reporting [2,[12][13][14][15][16][17][18][19][20]; it carries the potential to increase the accountability of those conducting clinical trial research; and it makes the identification of gaps in the health research landscape possible, thus facilitating priority setting in research [21][22][23][24][25][26][27].
The degree to which registered trial data can be used for these purposes depends on the completeness and meaningfulness of the data registered. The quality of data in registered records has been shown to be poor in the past. [2,14,15,[28][29][30][31][32][33][34][35][36][37][38][39][40][41] However, clinical trials registration has matured in recent years. Trialists may have gotten better at registering. Moreover, registries are likely to have improved their registration systems after the implementation of the International Standards for Clinical Trial Registries in 2010. [42] This study was conducted to investigate whether the poor quality of registration observed in the past has been due to trial registration being in its nascence, or whether it is a more persistent problem. To do so, we repeated a study conducted by us in 2009, using the same methods and the same research team. [2] Methods A random sample of 400 registered records of clinical trials that were registered between 1 January 2012 and 1 January 2013 was taken from the ICTRP database. Records of trials that were registered as having an observational study design were not eligible for the sample. For trials that were registered in more than one registry (duplicate records), only the record with the earliest registration date was considered eligible. [43] At the time the sample was taken the database included trials registered in fifteen different registries. [9] Sample size calculation A sample size of 380 records was chosen, to ensure that all upper and lower 95% confidence intervals for extrapolation to the entire ICTRP dataset, calculated using the Wilson score interval (see further under analysis), would deviate 5% at most from the estimated number. A sample size of 380 also fulfilled this study's requirements to detect relatively minor changes in the quality of the three primary outcomes: the quality of contact details, interventions and outcomes (minor changes were defined as a 10% increase or decrease in the proportion of adequately registered records). It allowed for detecting an increase or decrease of 10% (using two-tailed test and a = 0.05) with b.0.85 in the quality of contact details and interventions and with b.0.95 in the quality of primary outcomes.
In our previous study in 2009, 3% of trials were incorrectly registered as interventional. [2] Therefore, a final sample size of 400 records was chosen to allow for exclusion of these trials.

Data extraction
Registry name, trial ID, target sample size, inclusion criteria for gender and age of participants, recruitment status, date of registration, date of first enrolment and the public and scientific title for each record were downloaded from the ICTRP database and imported into Excel on 13 February 2013. Records were checked for the presence of entries in each of these fields.
All information that had to be extracted manually from the registered records was collected between 13 February 2013 and 23 February 2013. Information was always extracted manually from the complete registered record in the source registry.
Descriptive information on study design was extracted manually. Data on interventions and sponsorship was also extracted manually and was then coded. The system used to code interventions was adapted from the codes used for intervention types on ClinicalTrials.gov. [44] Primary sponsors were coded as being foundation, government, industry, university/hospital, or other. Trials were coded as being industry funded (primary sponsor was industry), partially industry funded (primary sponsor was non-industry, but secondary sponsor or source of monetary or material support was industry), or non-industry funded.
Records of trials that were registered as interventional but, during manual data extraction, turned out to be records of observational trials, diagnostic accuracy trials or treatment protocols for continuation of treatment after inclusion in a study protocol were excluded from further data extraction.
Descriptive statistics were generated for registry name, primary sponsor category, intervention type, study phase, study design, target sample size, randomization status and inclusion criteria for gender and age of participants. Registration dates and dates of first enrolment were compared to assess the degree of retrospective registration.
Contact information. The presence or absence of the following contact details was evaluated: name of a contact person (investigator or other), email address and telephone number. The WHO 20-item Trial Registration Data Set requires registration of separate scientific and public contact details.
[45] There was, however, variation in registration formats for contact details between different registries. Some registries had one field for contact details, others had two separate fields for public and scientific contact details and others multiple contact fields. For records with only one contact field the presence of contact information was extracted from that field. For records with multiple contact fields, if the contact details were present in any of the fields, the information was denoted to be present.
Interventions. Given the considerable variability in the types of interventions evaluated in trials, comparison of registration quality between different intervention categories is difficult. Therefore, the evaluation of the quality of registered intervention data was limited to trials that investigated drugs, biologicals or vaccines, including active comparators. Placebo comparators were not evaluated. For each intervention and active comparator the presence or absence of the following five intervention specifics was collected: name, dose, duration of the intervention, frequency of administration and route of administration. All intervention arms were assessed separately. Name was denoted to be present if a company serial number or a drug name was provided. Only interventions and active comparators mentioned in the intervention field were assessed. Other texts in the record were scanned for additional information on mentioned interventions. To assess the overall completeness of registration of intervention specifics, a binary outcome variable was used that could be incomplete versus complete registration of the intervention. Complete registration entailed the reporting of drug name, dose, duration, frequency and route.
Outcome measures. The number of primary outcomes per record was collected. Each primary outcome was evaluated for specificity, using a classification system adapted from the system used by Zarin et al in their assessment of quality of outcomes. [30,37] If a record contained multiple outcomes, all were assessed separately. Outcomes were classified as being a specific measure, a domain, vague, an unexplained abbreviation, or a part of safety monitoring.
Besides assessing the specificity of each outcome, the presence or absence of a time frame was collected for every outcome. Some outcomes assessed the duration of an event, the time to an event or were safety monitoring outcomes. For these outcomes, reporting a time frame is not possible, and the timeframe was therefore denoted as irrelevant. Time frames were denoted to be not meaningful when they did not specify a point in time when the outcome was to be measured.
Only outcomes mentioned in the outcome fields were assessed. Other texts in the record were scanned for additional information on mentioned outcomes.
To assess the overall quality of registration of primary outcomes, a binary outcome variable was used that could be registration of a specific measure with a meaningful time frame present or for which a time frame was irrelevant, versus any other outcome.
Finally, when there was more than one intervention (or active comparator) arm registered for a trial, or when there was more than one primary outcome registered, all intervention (and active comparator) arms and primary outcomes were assessed in this study. Multiple intervention arms and primary outcomes from one registered record are not independent. The effects of this nonindependence on our reported outcomes are expected to be limited.

Internal inconsistency in study design
Internal inconsistencies in study design fields were identified. [46] Internal inconsistencies were defined as records with multiple descriptors that were not compatible, such as ''singlegroup'' and ''controlled or randomized''; ''open-label'' and ''blinded''; and ''double-blinded'' without subject or investigator blinding.

Assessment rules
The assessment rules and methods for data extraction for this study are analogous with the rules and methods used in our previous study on the quality of registration. [2] As then, all records were assessed for eligibility by RV who then extracted and coded the data. A more detailed overview of the rules used in all assessments is provided in the supporting file that accompanies our previous publication. [2] Analysis 95% confidence intervals (CI) were calculated for proportions of trials in the samples using continuity corrected Wilson score intervals with Singleton et al. adjustments for finite populations. [47][48][49] These 95% CIs reflect the confidence with which these proportions, measured in our samples of records, predict true proportions in the overall populations of all interventional trials on the ICTRP. The quality of registration was compared between trials registered between 17 June 2008 and 17 June 2009 [2] and trial registered between 1 January 2012 and 1 January 2013 using the Newcombe-Wilson test with continuity correction (with a = 0.05). [47][48][49] This study intended to analyse changes in the quality of registration across all registries from 2008/2009 to 2012. However, the distribution of clinical trials across the registries changed from the former dataset to the latter, and several new registries were added to the ICTRP database. To be able to draw conclusions about changes in the quality of registration among the registries that were included in our first study, we conducted a sensitivity analysis for changes in data quality in the registered records from the three largest registries from 2008/2009 (ClinicalTrials.gov, ISRCTN and ANZCTR).
Statistical analyses were performed using MS Excel and SPSS 20.

Results
A sample of 400 records was taken from a total of 23,046 unique interventional trials that were registered between 1 January 2012 and 1 January 2013. 14 records were excluded from data extraction because the corresponding trials were of an observational or diagnostic accuracy study design or were a treatment protocol for continuation of treatment after inclusion in a study protocol. A total of 386 records was included for data extraction, of which 221 (57.3% [52.2%-62.2%]) investigated drugs, biologicals or vaccines ( Figure 1).
Baseline data on registry name, primary sponsor category, intervention type, study phase, study design, randomization status and inclusion criteria for gender of participants are presented in Table 1.
Records were additionally checked for the presence of entries in the fields for recruitment status, date of first enrolment and the public and scientific title. The former three were present in all records, the latter was reported in 379 records ( mentioned inclusion of participants ,18 years of age. Finally, registration dates and dates of first enrolment were compared. The majority of records in our sample did not provide a day for the date of first enrolment but only a month and a year, which limited this analysis to comparing the month in which trials were registered to the month in which the first participant was recruited. The registration date was in a later month than the date of first enrolment in 185 records ( (Table 2). Improvement in the presence of a name of a contact person was not significant. All changes for the subcategories of industry, non-industry and partially industry sponsored records were not significant. Contact details remained present less frequently among industry sponsored trials than among non-industry sponsored trials.

Quality of registration of contact information
The presence of contact details was disaggregated according to trials' recruitment status ( Table 3). The presence of names of contact persons did not differ markedly for trials with a different recruitment status, but email addresses, telephone numbers or either were present more frequently among recruiting or not yet recruiting trials than among completed trials, especially for industry sponsored trials.
Sensitivity analysis among the three largest registries showed effects that were congruent with the changes found in the full dataset. From 2008/2009 (693 trials) to 2012 (260 trials), reporting improved from 79.9% to 86.2% for the name of a contact persons, from 57.9% to 61.9% for an email address, from 62.5% to 67.7% for a telephone number, and from 67.2% to 70.8% for the presence of either.

Quality of registration of interventions involving drugs, biological or vaccines
There were 221 records of trials that investigated drugs, biologicals or vaccines. These reported 351 experimental or active comparator arms ( The degree of specificity of reported outcomes was assessed (Table 5). To prevent skewing of the data, the outcomes in the record with 52 outcomes were counted as one for this analysis (the 2 nd highest number of outcomes in any record was 12). 377 primary outcomes (57.6% [53.8%-61.4%]) were specific measures for which a meaningful time frame was present or for which a time frame was irrelevant. This constituted a significant improvement from the observed 38.2% in 2008/2009. Sensitivity analysis also showed improvements for the quality of reported primary outcomes in registered records from the three largest registries. From 2008/2009 (1186 primary outcomes) to 2012 (401 primary outcomes), the proportion of primary outcomes that were specific measures for which a meaningful time frame was present or for which a time frame was irrelevant improved from 38.5% to 66.1%.

Internal inconsistencies in study design
Internal inconsistencies in the study design fields were encountered in 10 records (2.6% [1.3%-4.9%]). This was a significant improvement from the observed 9.3% in 2008/ 2009. [46]

Differences between registries
Differences between registries in the quality of information on contact details, interventions and primary outcomes were assessed (Table 6). Only registries with more than ten records, intervention arms or outcomes, respectively, were included for this comparison. There are differences between registries in the quality of reporting, yet there are few that score good on all aspects of quality, or bad on all aspects.
To learn more about how data recording formats might influence data quality, data recording formats for contact details, interventions and primary outcomes were denoted for each of the registries that provided data to the WHO ICTRP at the time of the study (Table 7).

A persistent problem
This study was conducted using the same methods and the same research team as our previous study on the quality of registration. [2] There have been small but significant improvements in the quality of registration since 2009. However, important problems with quality remain and continue to constitute an impediment to the meaningful utilization of registered trial information.
There have been small improvements to the presence of contact details overall. This is partially due to the larger proportion of nonindustry trials in the analysis of trials registered in 2012, which do better on registering contact details. But across all sponsor categories quality also improved, the main exception being the continued lack of mentioning of names of contact persons by industry sponsors. Explicit mentioning of the name of the principal investigator is important to increase the accountability of trialists. Furthermore, despite improvements, contact information such as a telephone number or email address often remain absent. Remarkably, trialists appear to remove contact details when trials have been completed or stopped, in particular for industry sponsored trials. To allow patients, healthcare workers and other researchers to inform themselves of clinical trials, it is important that trialists can be contacted at any stage of a trial. Such information should remain available after a trial is completed or stopped.
There was some improvement in the completeness of intervention specifics for drug trials, however, the improvement was minor. Contrariwise, the improvement in the quality of registered outcomes was marked. This is a hopeful development for systematic reviewers, since in the absence of a complete trial protocol, registered clinical trial data constitute the only way to identify selective reporting. [2,13,[15][16][17][18][19] However, specific information about the outcome in registered records is necessary to detect selective outcome reporting as part of systematic reviews, and still almost half of all primary outcomes do not constitute a specific measure with a meaningful timeframe. Moreover, it has been proposed that the specificity of outcomes should be assessed at a greater level of granularity, to take into account more subtle forms of selective reporting. [30]   Other sponsors consisted of persons that were registered as primary sponsor, non-governmental organizations, collaborative research institutions and clinical research organizations. 3 Overlap was possible, total in this category was greater than 731 in 2008/2009 and greater than 386 in 2012. 4 Genetic interventions consisted of gene transfer therapy and somatic cell transplants. 5 The presence of study phase in records was analysed separately for trials in drugs, biologicals or vaccines. Finally, a very large percentage of records remains registered retrospectively, as has also been concluded by others. [50] Without prospective registration, before enrolment of the first participant, we cannot be certain that trial outcomes are not retrospectively registered in such a way that favours a particular result.
In conclusion, there have been small improvements to the quality of registered trial data, but poor quality is a persistent problem. Recent publications have also shown concomitant results reporting at individual registries to be problematically incomplete, such as at ClinicalTrials.gov [51][52][53], despite legal obligations in the US to report the findings of trials. [51,54] The causes of poor quality (and learning from other registries) The persistent nature of poor quality of registered clinical trial data suggests one or more pervasive causes. Although trialists themselves have a responsibility to ensure that the information in registered records is complete and accurate, registries can encourage high-quality registration through quality control processes and appropriate data recording practises. [2] Both are addressed in the International Standards for Clinical Trial Registries. [42] Our analysis suggests that there are important differences between registries with regards to registration quality. Notably, there are few that score badly on all three aspects of quality that we tested, or well on all. Rather, there are differences depending on which aspect is assessed, as becomes clear from Table 6, and from our sensitivity analyses, which showed that the three largest registries score better on intervention and primary outcome quality, but worse on the presence of most contact details. One explanation for these differences is the variation in data recording formats between registries. [31] For example, some registries specifically ask trialists for the methods of measurement for each outcome. Others have only free text fields for outcomes. Some registries ask for specific details on interventions, others, again, have only free text fields. Some registries ask trialists to categorize interventions and outcomes, others do not. For data quality and data aggregation purposes, [23] it is important that discrete options are offered where there is a limited set of possible answers (supplemented by a free text field to allow for additional explanation where needed), that different sub-aspects of data set    items are specifically queried (Table 7), and that the data recording formats are harmonized across all individual registries. A second explanation for the differences in the quality of registration between registries is the level of quality control that registries apply.
The differences in the quality of registration of different data items found in this study suggest that registries can learn from each other. Differences between registries in terms of data recording formats and their consequences for data quality deserve to be studied in more detail, so that registries can improve their formats based on the lessons from other registries. Registries could also draw lessons from each other about quality control, for example with regards to the information that is considered mandatory and a precondition for registration, and the different tiers of data checking (e.g. automated checks and manual checks [30,42]) that can be implemented to detect incomplete or non-meaningful entries. The International Standards for Clinical Trial Registries state that benchmarking of registries should be one of the next steps in standards development for registries. [42]

Enforcement
To be able to make use of the potential benefits that clinical trials registration offers, it is of paramount importance that registration is complete and accurate. However, it must also be comprehensive. [2] Enforcement of clinical trials registration has increased substantially over the past decade, [55] owing to national legislation on registration [4,30,56], policies by journal editors and publishers making registration a prerequisite for publication [1,[5][6][7]57], ethics committees and national research ethics oversight agencies requiring registration as part of procedures for ethics approval [3,55,58], policies by funders making registration a prerequisite for grant approval [59], international codes of research practice that recommend trial registration, such as the SPIRIT 2013 and CONSORT 2010 statements which include sections recommending the admission of trial registration details to both clinical trial protocols and reports [60,61], international codes of research ethics, such as the declaration of Helsinki [62], and self-regulation by universities [55] and the pharmaceutical industry [8]. Despite these measures, a proportion of trials currently remains unregistered, especially in countries lacking legislation on trial registration. [63][64][65][66][67] National legislation is crucial in enforcing the registration of all clinical trials. [4] Several of the other enforcement measures outlined above have been instrumental in creating momentum for clinical trials registration, such as journal and ethics review board requirements for registration, yet not all journal editors require registration as a pre-condition for publication, [67] not all clinical trials are conducted with the goal of publication, [4] and not all ethics committees have policies on clinical trials registration in place [58]. Therefore, it is imperative that all countries that have not implemented legislation on trial registration do so. [4] Furthermore, it is important that the remit of legislation on registration should cover all possible clinical trials, as is being recognized in the US and the EU. [68,69] Currently, in those   countries where legislation to enforce registration is present, its remit is often limited to a sub-set of trials. [30,68] With regards to enforcement, the commitment of the pharmaceutical industry to clinical trials registration is important and the past development of a Joint Position of several pharmaceutical associations on the disclosure of clinical trial information via clinical trial registries and databases is laudable. [8] However, the Joint Position needs revisiting on three important aspects. First, currently, it allows for registration after commencement of patient enrolment. This is in contradiction with policies on clinical trial registration by WHO and the International Committee of Medical Journal Editors (ICMJE). [1,2] Second, it allows trialists to withhold data specified by the WHO Minimum Trial Registration Data Set if they consider it sensitive. This, too, is in contradiction with policies on clinical trial registration by WHO and the ICMJE. [1,2] Third, the Joint Position mentions that ''registration of clinical trials on any one of a number of free, publicly accessible, internet-based registries should achieve the intended objectives''.
To ensure the quality of registered trial data, the WHO ICTRP search portal only provides access to data from trials registered at registries that meet certain quality standards (excluding, for example, registries managed by for-profit agencies). [70] To realize a single point of access to all clinical trial data conducted globally, it is important that the pharmaceutical associations include a commitment to registration in WHO approved registries in the next update of their Joint Position, as the ICMJE already has. [7] Finally, enforcement of trial registration by the pharmaceutical industry would be further advanced if support for clinical trials registration and results reporting would not be limited to statements from the pharmaceutical associations, but if more individual pharmaceutical companies would subscribe to the AllTrials campaign, following the example of GlaxoSmith-Kline. [71] Besides increasing the number of trials that is registered, enforcing measures could also help improve the quality of registration. Journal editors, for example, have been called upon to not only enforce registration itself, but to also implement quality control procedures. [19] Although editors have made clear that trial registration with missing or uninformative fields for the minimum data elements is inadequate, [1,5,57,72] little is known about to what degree journals are putting such measures into practice. [6] Similarly, both in the EU and in the US legislature supports the WHO Minimum Trial Registration Data Set -the minimum amount of trial information that must appear in a register in order for a given trial to be considered fully registered. [73,74] Failure to comply with registration legislation may result in penalties or withholding of federal grants. [54] Yet, little is known to what extent legislators are planning to invoke such measures, and whether the quality of registration could play a role in such decisions. For both legislators and journal editors discussion needs to be initiated on how far measures should go to discourage incomplete or inadequate registration. This applies to both the initial registration of a clinical trial, which was the subject of this study, as for results reporting in registry databases. [51][52][53]

Conclusion
There have been small but significant improvements in the quality of registration since 2009. However, important problems with quality remain and continue to constitute an impediment to the meaningful utilization of registered trial information. More effort needs to be made to improve data recording formats, enhance quality control measures and scale up enforcement of trial registration.   Table 7. Data recording formats for the three primary outcomes of this study (contact information, intervention specifics and outcome quality) at the registries that provided data to the ICTRP at the time of the study in 2013.