Methodological Reporting of Randomized Trials in Five Leading Chinese Nursing Journals

Background Randomized controlled trials (RCTs) are not always well reported, especially in terms of their methodological descriptions. This study aimed to investigate the adherence of methodological reporting complying with CONSORT and explore associated trial level variables in the Chinese nursing care field. Methods In June 2012, we identified RCTs published in five leading Chinese nursing journals and included trials with details of randomized methods. The quality of methodological reporting was measured through the methods section of the CONSORT checklist and the overall CONSORT methodological items score was calculated and expressed as a percentage. Meanwhile, we hypothesized that some general and methodological characteristics were associated with reporting quality and conducted a regression with these data to explore the correlation. The descriptive and regression statistics were calculated via SPSS 13.0. Results In total, 680 RCTs were included. The overall CONSORT methodological items score was 6.34±0.97 (Mean ± SD). No RCT reported descriptions and changes in “trial design,” changes in “outcomes” and “implementation,” or descriptions of the similarity of interventions for “blinding.” Poor reporting was found in detailing the “settings of participants” (13.1%), “type of randomization sequence generation” (1.8%), calculation methods of “sample size” (0.4%), explanation of any interim analyses and stopping guidelines for “sample size” (0.3%), “allocation concealment mechanism” (0.3%), additional analyses in “statistical methods” (2.1%), and targeted subjects and methods of “blinding” (5.9%). More than 50% of trials described randomization sequence generation, the eligibility criteria of “participants,” “interventions,” and definitions of the “outcomes” and “statistical methods.” The regression analysis found that publication year and ITT analysis were weakly associated with CONSORT score. Conclusions The completeness of methodological reporting of RCTs in the Chinese nursing care field is poor, especially with regard to the reporting of trial design, changes in outcomes, sample size calculation, allocation concealment, blinding, and statistical methods.


Introduction
The randomized controlled trial (RCT) is the most appropriate design for evaluating the effectiveness of a nursing intervention among eligible primary study designs for evidence-based practice [1]. Evidence-based nursing practitioners believe that randomization and unbiased allocation of interventions assure methodological quality and the subsequent findings of RCTs [2,3]. However, well-conducted trials may be reported poorly [4], and failing to report all of the study's methods, details, and findings, especially for RCTs, seriously distorts the evidence base for clinical decisionmaking [5].
As a set of evidence-based reporting guidelines for RCTs, the Consolidated Standards of Reporting Trials (CONSORT) [6] was developed to improve the transparency of reporting and interpreting RCTs, minimize biased conclusions, ultimately facilitate clinical decision making, and indirectly drive better design of trials. The CONSORT guidelines have been associated with improving the quality of RCT reporting [7]. To date, CONSORT has been accepted by 600 medical journals, including 20 nursing care journals in English [8], and it has been revised twice, in 2001 and 2010 [9].
For RCTs in various disciplines (e.g., diagnostic procedures, interventions, cancer trials, and alternative medicine), the reporting quality has been assessed via CONSORT [5] to ensure their transparency, and found widespread inadequate reporting of methodological issues. However, evidence is still deficient on the reporting quality of RCTs in nursing care, especially in China. Existing reviews have focused on only a specific discipline [10,11], were not conducted systematically [12,13], used a modified CONSORT checklist according to the purposes of the study [14], or focused on only a specific methodological issue [15,16].
This study was thus conducted to investigate (1) the adherence to methodological reporting complying with CONSORT on Chinese nursing RCTs, and (2) the correlation between CONSORT adherence and key indicators regarding characteristics and methodological factors.

Information Source
According to their total number of citations and impact factors (IFs) in the ''Chinese S&T Journal Citation Reports 2012'' [17], we selected the following five leading Chinese nursing journals(see Appendix S1): Chinese Journal of Nursing (CJN, 1954-), Chinese Journal of Modern Nursing (CJMN, 1995-), International Journal of Nursing (IJN, 1980-), Chinese Journal of Practical Nursing (CJPN, 1985-) and Journal of Nurses Training (JNT, 1986-). In June 2012, a comprehensive electronic search strategy for the Chinese Bio-Medicine disc (CBM; see Appendix S2) was performed using ''randomized controlled trial'' or ''clinical trial'' as keywords, limited to these five targeted journals.

Eligibility Criteria
Inclusion criteria.
1) Citations claiming to be RCTs in the titles and/or abstracts, according to the ''Cochrane definitions and criteria for randomized controlled trials and controlled clinical trials'' [18]. 2) Controlled trials with two comparators.
3) Trials of both nursing interventions (e.g., drug interventions, social and psychological interventions, traditional Chinese medicine interventions, complex interventions) and nonnursing interventions (e.g., nursing education program interventions).
1) ''Quasi-random'' trials: when ''random'' was mentioned with a high-risk randomization method, such as alternation, assignment based on date of birth, case record number, or date of presentation. 2) Irrelevant study reports: reports that are published in a summary form, reviews of randomized trials, and randomized trials on animals, even though they may be reported as ''random.'' 3) Other study designs, such as cohort study and case series report.

Study Selection and Data Extraction
All citations from the CBM were allocated evenly to three units. By first screening titles and abstracts and then screening fulltexts, three groups consisting of two authors each selected eligible studies from these units of citations independently. When a disagreement arose in any of groups, another group would participate to arrive at an agreement. Then, two authors extracted relevant information and data from the included studies. A well-designed extraction form (Appendix S3) was formulated to assess the reporting quality of RCTs according to CONSORT [6]. The extraction form included (1) general characteristics [5] (i.e., journal name, IF, publication date, number of authors, multicenter trials, types of interventions, sample size, and funding source) and a description of the interventions [19], including length of descriptions of interventions and controls in words and the reproducibility of the interventions, which authors evaluated by assessing whether the intervention was described in enough detail to be reproducible; (2) CONSORT methodological items; and (3) methodological quality measurements [20], including allocation concealment and intention-to-treat (ITT) analysis. The overall CONSORT methodological items score was the total score of 17 items with response options of ''yes'' (scored as 1) or ''no'' (scored as 0) on whether the component had been reported as recommended. Items were scored as ''no'' (i.e., 0) if the item was only partly met by the research report. Disagreements during data extraction were resolved by discussion.

Quality Control
We implemented rigorous quality control throughout the study. Before selecting the eligible studies, investigators were trained to distinguish RCTs from irrelevant study designs using the definitions from the Centre for Evidence-Based Medicine, Oxford, UK (University of Oxford) [21]. Before data extraction, the extraction form (Appendix S3) following CONSORT guidelines was established via three rounds of pre-survey and team discussion.

Data Analysis
Using SPSS 13.0, we performed a descriptive analysis, obtaining frequencies and percentages for categorical variables, and means, medians, standard deviations, and inter-quartile ranges (IQR) for continuous variables. Then, exploratory regression was conducted to examine whether general and methodological characteristics are associated with reporting quality, as measured by the CONSORT methodological items score. Univariate regression was performed for each potentially associated variable, and variables with P values lower than 0.25 were then included in a stepwise backward linear regression. Any P value lower than 0.05 was regarded as statistically significant. In this study, the sample sizes of the trials were included to power the exploratory linear regression.

Study Inclusion
In total, 58,823 citations published in the five leading nursing journals from 1954 to June 2012 were retrieved via the ''Journal Search Engine'' function of the CBM disc. After screening abstracts and fulltexts, 5519 fulltexts that mentioned ''random'' were initially identified. Among them, 5502 trials described ''randomization'' in both abstracts and fulltexts, and 17 described ''randomization'' in only the fulltexts. Six hundred and eighty trials met the criteria for ''details of randomization'' and were included in the final analysis. Figure 1 depicts a flow diagram of the inclusion decisions.

Characteristics of Included Studies
Most of the included trials (99.9%) were published after the 2001 CONSORT (Appendix S4). Furthermore, 27.6% (188/680) of trials were published in CJN and JNT, which had higher IFs than the average (mean 6 SD = 1.2061.05); 61% (415/680) of trials were reported by no more than three authors (i.e., the median number of authors). Two point five percent (17/680) of trials reported a collaboration between multicenter studies. However, 21.6% of these trials did not specify the number of clinical trial centers. Most trials (83.4%) studied the implementation of non-pharmacological interventions. The median sample size of the included studies was 90 (IQR = 62.75). Most trials (75.6%) did not mention financial support. By our judgment, 85.4% of trials provided enough details in the description of interventions to facilitate replication; however, only 48.2% of control group protocols were sufficiently described. Nisnety percent of trials from CJPN described the control group protocol as ''the control group used standard nursing inventions for this disease.'' There were significant differences in the number of words used to describe the interventions and controls (median = 215.0 and IQR = 422.75 for interventions, and median = 32.0 and IQR = 47.0 for controls, Z = 221.192, P,0.001). For methodological quality measurements, only two trials (0.3%) reported the details of the allocation concealment mechanism; 85.3% of trials had no dropouts, and 5.7% conducted per protocol analysis. Table 1 depicts the characteristics of the included studies.

Reporting of CONSORT Methodological Items
When screening abstracts rather than fulltexts, 97.9% of RCTs could be identified. Only two trials (0.3%) reported ''randomized control trial'' in the title. Overall, the mean CONSORT methodological score was 6.3460.97 out of 17 sub-items. Additionally, in order to explore the influence of the score of ''blinding (11a, 11b)'', an analysis excluding data on blinding showed that the mean and SD of the CONSORT score were 6.28 and 0.94 respectively, which were similar to the mean score obtained while including data on blinding. No RCT reported descriptions and changes in ''trial design,'' changes in ''outcomes'' and ''implementation,'' or descriptions of the similarity of interventions for ''blinding.'' For the description of trial design, 99.7% of the studies used a parallel design while 0.3% used a crossover design. Sub-standard reporting was also found regarding the details of ''settings of participants'' (13.1%), ''type of randomization sequence generation'' (1.8%), calculation methods of ''sample size'' (0.4%), explanation of any interim analyses and stopping guidelines of ''sample size'' (0.3%), additional analyses of ''statistical methods'' (2.1%), and targeted subjects and methods of  ''blinding'' (5.9%). In addition, no central randomization that implied allocation concealment was reported. Only two studies conducted allocation concealment through sequentiallynumbered opaque sealed envelopes. More than 50% of trials described randomization sequence generation, the eligibility criteria of participants, ''interventions,'' and definitions of ''outcomes'' and ''statistical methods.'' Table 2 depicts the reporting of individual CONSORT methodological items.

Predictors Associated with CONSORT Methodological Score
We explored the association of predictors, including general characteristics and methodological quality measurements, with CONSORT methodological score with a linear regression and found that publication year and ITT analysis were weakly associated with CONSORT score. Each predictor was associated with a small increase of 0.047 and 0.601 points to the overall CONSORT methodological score. The two predictors associated with CONSORT score (i.e. publication year and ITT analysis) remained after the data on ''blinding'' were excluded, and their P values were 0.005 and ,0.001 respectively. Table 3 depicts the results of our linear regression.

Discussion
In this review, we investigated 680 RCT fulltexts using CONSORT methodological items and found that RCTs of nursing interventions in Chinese nursing care journals reported Parallel and crossover design are judged by authors. **In order to explore the influence of the score of ''blinding (11a, 11b),'' we excluded data on blinding and found the mean and SD to be 6.28 and 0.94, respectively, which are similar to the mean and SD obtained when including data on blinding. doi:10.1371/journal.pone.0113002.t002 ''trial design,'' ''participant settings,'' changes in ''outcome,'' ''sample size,'' ''allocation concealment mechanism,'' ''implementation,'' ''blinding,'' and additional analyses of ''statistical methods'' poorly. The overall CONSORT score was low. Weak correlations between CONSORT score and two variables (i.e., publication year and ITT analysis) were found.
Concordant with our findings, Zhang et al. [11] showed that the reporting of methodological items in cancer nursing care literature was also poor in terms of outcomes, sample size, randomization sequence generation, allocation concealment mechanism, implementation, and blinding. Zhu et al. [15] demonstrated that only 3.8% of RCTs of nursing interventions described randomization sequence generation methods; furthermore, in traditional Chinese medicine nursing studies [10], 7.8% of RCTs described randomization sequence generation, 1.4% described the allocation concealment mechanism, and 3.1% described blinding. Evidence from RCTs of nursing interventions in English [13,14,16] also supported our findings. Samaan et al. [5] reviewed systematic reviews of adherence to reporting guidelines across different clinical areas and found a prevailing lack of reporting quality in most disciplines. Additionally, Samaan et al. [5] showed that 80% of the studies that did not adhere to CONSORT guidelines had negative conclusions, such as ''some improvements are needed, reporting inadequate/poor/medium/suboptimal, etc.'' Our findings are consistent with other systematic reviews in other specializations [19,22]. Given these findings, we have provided important evidence to alert Chinese nursing researchers of the urgent need for improving the reporting of RCTs on nursing interventions. This is especially true for randomization, as we found that 87.7% (4839/5519) of RCTs that mentioned ''random'' in abstracts and/or fulltexts did not actually include details of the randomization or used incorrect randomized methods, such as alternate allocation and allocation by record number. Similar results were reported by Wu et al. [24].
We found poorer reporting of blinding in 83.4% of nonpharmacological interventions, but no correlation was established via the regression analysis. Due to some methodological challenges, such as the complexity of the intervention [23], expertise of the care provider [23], and difficulties with blinding [16], it is believed that non-pharmacological interventions always limit the blinding of participants and caregivers; however, the blinding of data collectors is often achievable. Regardless of whether blinding is possible, authors can and should state who was blinded. Therefore, although it seemed unnecessary in some included studies, blinding was judged as impracticable because the authors did not report its details. Our regression analysis indicated that publication year had a weak correlation with reporting quality. This demonstrates that reporting quality is improving yearly. We also examined the relationship between reporting quality and methodological quality of RCTs by regression of allocation concealment and ITT analysis. The regression indicated only weak correlations with ITT analysis, and no correlations with the other variable. In order to explore the influence of the score of ''blinding (11a, 11b)'' on regression analysis, data on ''blinding'' were deleted, and the results showed that the two same predictors (i.e. publication year and ITT analysis) were associated with CONSORT score analysis.
The main strengths of this study were the comprehensive search strategy, determination of associations of CONSORT score with various predictors, and the well-conducted quality control assessment. However, this review had some limitations. First, nursing interventions are often far more complicated than drug interventions are. We did not investigate the reporting of items related to external validity. Second, as our eligible criteria was not restricted to any specific nursing intervention such as drug interventions, social and psychological interventions, traditional Chinese medicine interventions, or complex interventions, we did not use CONSORT extensions for social and psychological interventions, non-pharmacological interventions, herbal interventions, and CReDECI criteria [25][26][27][28]. In addition, we did not find a pragmatic trial among the included studies, so the CONSORT extension for pragmatic trials [29] was also not used. Third, according to our eligibility criteria, no dissertation in nursing care

Conclusions
In conclusion, the completeness of methodological reporting of RCTs in Chinese nursing care journals was very poor, especially in terms of trial design, outcome changes, sample size calculation, allocation concealment, blinding, and statistical methods. Authors are highly recommended to use the CONSORT criteria when submitting reports of RCTs. Editors should also present their peerreviewed manuscripts under the guidance of CONSORT to ensure that journal articles contain the key clinical and methodological details that readers need. Initiatives recommended by Altman [30] should be accepted; these suggest that authors should sign a publication transparency declaration as part of every journal submission and journals should update their instructions to authors.