The quality of reporting in randomized controlled trials of acupuncture for knee osteoarthritis: A cross-sectional survey

Objective To assess the reporting quality of acupuncture trials for knee osteoarthritis (KOA), and explore the factors associated with the reporting. Method Three English and four Chinese databases were searched from inception to December 2016 for randomized control trials testing effects of acupuncture for knee osteoarthritis. We used the standard CONSORT (2010 version), CONSORT Extension for Non-Pharmacological Treatments, and STRICTA for measuring the quality of reporting. Using pre-specified study characteristics, we undertook regression analyses to examine factors associated with the reporting quality. Results A total of 318 RCT reports were included. For the standard CONSORT, ten items were substantially under-reported (reported in less than 5% of RCTs), including specification of important changes to methods after trial commencement (0.6%), description of any changes to trial outcomes (0.0%), implementation of interim analyses and stopping guidelines (0.6%), statement about why the trial ended or was stopped (1.6%), statement about the registration status (4.4%), accessibility of full trial protocol (4.7%), implementation of randomization (4.7%), description of the similarity of interventions (3.5%), conduct of ancillary analyses (3.8%) and presentation of methods for additional analyses (4.4%). Four of the STRICTA items were under-reported (reported in less than 10% of RCTs), including description of acupuncture style (8.5%), presentation of extent to which treatment varied (1.3%), statement of practitioner background (7.2%) and rationale for the control (9.1%). For CONSORT Extension, the reporting was poor across all items (reported in less than 10% of trials). Trials including authors with expertise in epidemiology or statistics, published in English, or enrolling patients from multiple centers were more likely to have better reporting. Conclusions The reporting in RCTs of acupuncture for KOA was generally poor. To improve the reporting quality, journals should encourage strict adherence to the reporting guidelines.


Conclusions
The reporting in RCTs of acupuncture for KOA was generally poor. To improve the reporting quality, journals should encourage strict adherence to the reporting guidelines.

Background
Randomized controlled trials (RCTs) are the gold standard for assessing the effects of health care interventions [1]. However, RCTs may yield misleading results if they lack methodological rigors [2]. Adequate reporting of RCTs is one of critical methodological issues, since the information reported has profound impact on the decisions by healthcare professionals and policy makers. Previous studies showed that RCTs with poor reporting, compared to those with good reporting, yielded larger effect estimates across a variety of healthcare conditions [3].
In order to improve the reporting of RCTs, scientific communities have made great efforts to develop recommendations, such as the Consolidated Standards of Reporting Trials (CON-SORT) statement which aims to improve the general reporting of RCTs [4,5]; the CONSORT extension for nonpharmacological treatments which addresses the reporting issues specific to complex interventions, such as surgery, devices, rehabilitation, psychotherapy, behavioral interventions, and complementary and alternative medicine [6]; and the Standards for Reporting Interventions in Clinical Trials of Acupuncture (STRICTA), a recommendation for the descriptions of acupuncture treatments [7].
Acupuncture is an important healthcare intervention. Recent years have seen burgeoning increase of RCTs testing effects of acupuncture. Compared to drug trials, acupuncture interventions are typically complex, patient recruitment is often difficult, and standardization of intervention is more challenging. Consequently, reporting of acupuncture trials is more complex, and careful and meticulous reporting is of paramount importance. Several studies have examined the issue of reporting among acupuncture trials, and identified a number of issues regarding inadequate reporting [8][9][10][11][12].
Nevertheless, none specifically examined the reporting of RCTs testing acupuncture knee osteoarthritis (KOA) [13]. As a traditional intervention, acupuncture has been widely used to treat osteoarthritis disorders in China as well as developed countries [14,15]. In the US, about one million patients used acupuncture to treat musculoskeletal disorders [16]; between 30% and 40% of general practices in England provide complementary treatment options for patients with KOA, among which acupuncture is the most popular choice [17]. Lack of adequate reporting of details in such trials would make the effective use of trial evidence less likely. Even in certain circumstances, this would lead to misled healthcare decisions. Therefore, we conducted a cross-sectional survey to specifically assess the extent to which the current RCTs examining acupuncture for KOA comply with the recommendations by the established reporting standards, and explore factors associated with the reporting.

Study selection
We included RCTs published either in English or Chinese as full-text reports that enrolled patients diagnosed with KOA, and compared acupuncture versus a control. We defined acupuncture as a stimulation of the body or auricular points regardless of the type of stimulation [9]. Any type of acupuncture was eligible for inclusion, such as electro-acupuncture, filiform needle, fire needle, silver needle, dry needle, laser acupuncture, ear acupuncture, and scalp acupuncture, regardless of the duration of treatment. The control may include acupuncture, pharmacologic intervention, placebo acupuncture (placing needle on the surface of skin without penetration), sham acupuncture (placing needle on sham points near acupuncture point), waiting list, and physical treatments (e.g. exercises, weight loss). RCTs that combined acupuncture with moxibustion were eligible, if using moxibustion as a co-intervention across groups.

Data sources
We searched PubMed, EMBASE and Cochrane Central Register of Controlled Trials (CEN-TRAL) and four Chinese Databases, including Chinese Biomedical Database (CBM), National Knowledge Infrastructure(CNKI), Wanfang and VIP, all from the inception to December 2016. The search terms were customized for each individual databases (S1 File). Reference lists of all eligible trial reports were screened for additionally eligible studies.

Study process
Two investigators (PLJ and JLL) independently screened titles and abstracts for potential eligibility. They subsequently read full texts of potentially eligible reports for final eligibility. Then, the two investigators (PLJ and JLL) independently assessed the quality of reporting of the eligible RCT reports. Any disagreements were resolved through discussion.

Data collection
We collected the information regarding study characteristics from each eligible RCT, as follow: name of first author, year of publication, journal name, journal type, sample size, number of groups, length of follow up, funding source (not-for-profit funding, for profit funding, clearly stated, not funded and not reported) and statistical significance of the primary outcome (p<0.05). When there was no clearly specified primary outcome or there was more than one primary outcome, we used the pre-specified criteria for selecting a primary outcome (S2 File) [18].
In order to measure the quality of reporting, we used the CONSORT Statement (2010 version) and the CONSORT Extension for Nonpharmacologic Treatments. The standard CON-SORT recommendations contain 36 items [5], and the CONSORT extension include 13 additional items [6]. We also used STRICTA to measure details specific to acupuncture (17 items) [7]. This resulted in a total of 63 items for the finial questionnaire. Each item was assigned one of the three response options: 'Yes' for compliance of reporting, 'No' for noncompliance, and 'NA' representing that the item was not applicable (S1 Table).
We developed data forms according to the checklists, and pilot-tested the forms by one author (PLJ). Then, a group discussion was undertaken to clarify the definition of each item. Thereafter, the assessment was calibrated using a random sample of 15 reports. Finally, data extraction was performed by two investigators (PLJ and JLL).

Data analysis
Descriptive statistics were used to summarize the characteristics of included studies. Dichotomous data were presented as number and percentage and continuous variables were described as median with interquartile range (IQR).
We summarized the score according to the checklist. Each item was given one point if reported by a study, otherwise zero point was given. The maximum possible scores for the standard CONSORT checklist, CONSORT extension, and STRICTA were 36, 10 and 17 points. We calculated the total score by adding each of the component checklists (i.e. standard CONSORT plus CONSORT extension and STRICTA) and the score for standard CONSORT (i.e. CONSORT score).
To examine the association of reporting quality with study characteristics, we pre-specified five factors. We initially listed potentially relevant factors based on our hypotheses and findings from previous reports. Then, the study team, consisting of clinical trial experts, statisticians and acupuncturists, discussed their relevance to our study. Our discussion ended up with five factors, including author's affiliations to the epidemiology or statistics department (yes vs no) based on the information included in the RCT reports, language (English vs Chinese), multi-center study (yes vs no), sample size (sample size 80 vs > 80, categorized according to the median) and significance of primary outcome (P < 0.05) (yes vs no).
We used univariable and multivariable linear regression analyses to examine the association of reporting quality with the pre-specified variables. We conducted two set of analyses, one for the overall score (i.e. standard CONSORT, CONSORT extension plus STRICTA), and one for the standard CONSORT score. We checked and assured that the scores did not appear to violate the assumption of normality.
In order to examine the impact of the scoring approach on the regression analysis, we conducted one sensitivity analysis, in which we assigned one point to an item if it was reported by the trial under assessment or not applicable. We then explored the association between the putative factors with the generated scores.

Study searching and selection
The search yielded 4,527 reports. After title and abstract screening, 557 reports were potentially eligible; upon reading full texts, 318 RCT reports proved eligible (Fig 1). The details of included RCTs were listed in S3 File.

Compliance of reporting to the standard CONSORT and CONSORT extension checklists
Among the 36 items of the standard CONSORT checklist, only four were adequately reported among those trials, including specification of study objectives or hypotheses (84.0%), statement about the eligibility criteria for participants (92.8%), description of study settings and locations (85.2%), and generalizability of the trial findings (87.7%) ( Table 2).
Poorly reported items (i.e. reporting in less than 5% of trials) were: specification of important changes to methods after trial commencement (0.6%), description of any changes to trial outcomes after the trial commenced with reasons (0.0%), implementation of interim analyses and stopping guidelines (0.6%), statement about why the trial ended or was stopped (1.6%), statement about the registration (4.4%), accessibility of full trial protocol (4.7%), implementation of randomization (4.7%), description of the similarity of interventions (3.5%), conduct of ancillary analyses (3.8%), and presentation of methods for additional analyses (4.4%).
All the ten CONSORT extension items were poorly reported (reported less than 10% of trials), such as description of the experimental treatment, comparator, care providers, centers and blinding status (2%), presentation of eligibility criteria for centers and those performing the interventions or statement about if the co-interventions were blinded to group assignment (0%) ( Table 2). Compared with the trials published in Chinese, those published in English journals were more likely to report items related to the implementation of randomization, allocation concealment and blinding, although the adequate reporting of the items were generally poor.

Compliance of reporting to the STRICTA checklist
The reporting about acupuncture inventions were seriously limited. Items with markedly incomplete reporting (reported in less than 20% of trials) were: description of style of acupuncture (8.5%), statement of reason for the treatment (14.2%), presentation of extent to which treatment was varied (1.3%), description of details of other interventions (18.6%), specification of setting and context of treatment (15.1%), statement of practitioner background (item 61, 7.2%) and rationale for the control (9.1%).

Factors associated with the reporting quality
Multivariable linear regression analyses showed that RCTs including authors with expertise in epidemiology or statistics (β coefficient 6.37, 95% confidence interval (CI) 3.49 to 9.24), published in English language (β coefficient 8.35, 95% CI 6.68 to 10.01), and enrolling multiple study sites (β coefficient: 4.80, 95% CI: 2.85 to 6.76) were statistically associated with a higher total score (indicating better overall reporting quality) ( Table 4). The beta coefficients of each individual variable corresponded to an increase of 4.80, 6.37 and 8.35 points to the total score, respectively. The analysis with the standard CONSORT score had similar results with the total score (Table 5). Sensitivity regression analyses suggested similar results (S4 File).

Discussion
Our study identified 318 RCTs of acupuncture for KOA. To the best of our knowledge, this is the first study that systematically assessed the extent to which such trials complied with the     reporting guidelines in this specific field. Across those RCTs, we found that a considerable number of items were not adequately reported, which may jeopardize the evaluation of internal and external validity of trial results. The apparent low adherence rate is primarily due to the poor reporting; However, the inapplicability of some items (e.g. description of any changes to trial outcomes, implementation of interim analyses and stopping guidelines and statement about why the trial ended or was stopped) to a small proportion of trials may affect the assessment as well.
Our study showed that less than half of the standard CONSORT items were reported. Reporting of items related to methodological domains, such as type of randomization, allocation concealment, and blinding was particularly incomplete. This is likely due to the poor reporting of Chinese RCTs, which accounted for 83% of total reports. Similar to previous report, the methodological quality of most acupuncture trials was generally poor in Chinese journals [15,43]. The results from our regression analyses also supported that RCTs published in Chinese had lower reporting quality. This finding is consistent with earlier studies addressing reporting quality in other subspecialties [9,11,38,[44][45][46].
Although the STRICTA statement recommends reporting on acupuncture rationale and practitioner background, information related to these items seems to have been largely under- reported. This is in line with the similar studies conducted by Lu and Ma [11,47]. Since acupuncture is a practitioner-dependent, non-pharmacological complex intervention, adequate reporting of items related to study contexts is essential for readers to determine whether the results of a study apply to their own practice [48]. We would argue that further detail is required for acupuncture interventions, which are often far more complex than drug interventions.
In contrast, our study found that the item related to details of needling materials was well reported (79.2% complete). This is inconsistent with the research published in 2013 that common missing element of non-pharmacological interventions in RCTs were materials [40]. This discrepancy may be because the included studies were limited to trials of acupuncture, and we included CONSORT extension and STRICTA that had items specific to acupuncture intervention [9]. The findings probably reflect increasing awareness and requirement of adopting STRICTA by authors, editorials and peer-reviewers. [48] The inconsistent and suboptimal reporting across items implied that certain items may have been treated differentially in their importance to authors [9]. Removal of details about interventions (i.e. those related to STRICTA recommendations) from main reports, due to word limitations or suggestions from editors and peer reviewers, might be an external issue limiting the trial reporting [48]. One study showed that the page length was associated with reporting quality [49].
We identified that the author expertise with epidemiology or statistics was associated with higher quality trials. This finding was consistent with an earlier study, in which inclusion of an author who had a cited degree in epidemiology or statistics was almost three times (odds ratio = 2.9) as likely to be of higher quality [50]. Our study also found that trials published in English journals were associated with better reporting [15]. This highlighted that Chinese journals should strictly adopt reporting guidelines, so that transparent, complete and accurate reporting of RCTs can be achieved [49]. Our regression analyses also showed that multicenter studies had better reporting quality. A similar finding was also reported that if RCTs are larger and involve more patients and centers, the reporting quality was improved [51].

Strengths and limitations
Our study has some strengths. We systematically examined the extent to which randomized trials examining acupuncture intervention for knee osteoarthritis adhered to the CONSORT statement and STRICTA guidelines. We conducted a comprehensive search, developed explicit eligibility criteria, applied rigorous methods for screening studies and collecting data, and used the widely accepted checklists, including the CONSORT statement and STRICTA guideline for assessing quality of reporting.
Our study also has limitations. In assessing quality of reporting, we calculated a quality score, assuming equal weight of each item, although the items of those checklists may carry varying weights (which were yet to be established). We did not use the latest version of the CONSORT extension for non-pharmacological treatments because it was not available when conducting this study. Third, the present study did not investigate the reporting quality of trials published in languages other than Chinese and English; we expected that such studies were very few. Fourth, when scoring the reporting of the items, some items may not be applicable to all trials. For example, those items such as clustering may not be appropriate for a single center comparison; and reporting of interim analyses is not applicable if a trial was not planned such analysis. These situations may have affected our comparison. Nevertheless, both the main and sensitivity analyses showed consistent findings, suggesting that our results are robust across the coding systems.

Conclusion
The reporting in RCTs of acupuncture for KOA was generally suboptimal. Items related to methodology, acupuncture rationale, practitioner background and comparator interventions remained under-reported in many trials. To improve the reporting quality, journals, especially those published in Chinese, should encourage strict adherence to the CONSORT and STRICTA guidelines.