Quality of Reporting of Randomised Controlled Trials of Herbal Interventions in ASEAN Plus Six Countries: A Systematic Review

Background Many randomised controlled trials (RCTs) of herbal interventions have been conducted in the ASEAN Communities. Good quality reporting of RCTs is essential for assessing clinical significance. Given the importance ASEAN placed on herbal medicines, the reporting quality of RCTs of herbal interventions among the ASEAN Communities deserved a special attention. Objectives To systematically review the quality of reporting of RCTs of herbal interventions conducted in the ASEAN Plus Six Countries. Methods Searches were performed using PubMed, EMBASE, The Cochrane Library, and Allied and Complementary Medicine (AMED), from inception through October 2013. These were limited to studies specific to humans and RCTs. Herbal species search terms were based on those listed in the National List of Essential Medicines [NLEM (Thailand, 2011)]. Studies conducted in the ASEAN Plus Six Countries, published in English were included. Results Seventy-one articles were identified. Thirty (42.25%) RCTs were from ASEAN Countries, whereas 41 RCTs (57.75%) were from Plus Six Group. Adherence to the recommended CONSORT checklist items for reporting of RCTs of herbal interventions among ASEAN Plus Six Countries ranged from 0% to 97.18%. Less than a quarter of the RCTs (18.31%) reported information on standardisation of the herbal products. However, the scope of our interventions of interest was limited to those developed from 20 herbal species listed in the NLEM of Thailand. Conclusions The present study highlights the need to improve reporting quality of RCTs of herbal interventions across ASEAN Plus Six Communities.


Introduction
Traditional Medicine is recognised as part of historical and cultural heritage in the communities of the Association of Southeast Asian Nations (ASEAN) Member States. The role and contribution of traditional medicine has been highlighted by the ASEAN Socio-cultural Community (ASCC) Blueprint under section B4: Access to healthcare and promotion of healthy lifestyles [1]. Under which, the strategic objectives were to ensure access to adequate and affordable healthcare, medical services and medicine, and promote healthy lifestyles for the people of ASEAN. Specific actions include the facilitation of research and cross-country exchange of experience in promoting the integration of safe, effective and quality Traditional Medicine, Complementary and Alternative Medicine (TM/CAM) into the national healthcare system, notably, the strengthening of the evidence base for herbal medicines and products.
ASEAN's external relations with other nations led to the formation of an economic partnership known as the ASEAN Plus Six Group, which comprised of the members of the ASEAN plus Australia, China, India, Japan, New Zealand, and South Korea. This regional framework signifies the promotion of cooperation, prompting economic ties, increasing market scale and resource supply capacity [2].
It has been demonstrated that randomised controlled trials (RCTs) of herbal interventions seldom provide adequate methodological information, and the quality of reporting is poor [3][4][5]. Gagnier and colleagues (2006) studied the quality of reporting of RCTs of herbal medicine interventions, and observed that less than half (45%) of the consolidated standards of reporting trials' (CONSORT) checklist items was reported across the 206 included RCTs [5]. In addition, many study reports overlooked the importance of the detailed information on the herbal products being investigated, and often, specific characteristics of the herbal interventions were omitted [6].
The significance of adequate and transparent reporting of herbal interventions cannot be overstated as it is necessary to elicit clinical significance. An Elaborated CONSORT Statement was developed to provide recommendations for the reporting of herbal medicine trials [7], which serve as an aid to editors and reviewers in assessing the internal/external validity and reproducibility of herbal medicine trials, thereby an accurate assessment of safety and efficacy can be achieved [8].
The purpose of the current study was to strengthen the evidence base for herbal medicines and products specific to the ASEAN Community. To date, no systematic assessments of the quality of reporting of RCTs of herbal interventions in ASEAN Communities have been published. Adequate reporting is considered crucial for the audience of the reports to reliably interpret the outcomes of the trials. In order to strengthen the evidence base for herbal medicines specific to the ASEAN Community as outlined in the ASCC Blueprint, specific study within the Community of ASEAN Plus six is called for. ASEAN Plus Six allowed for larger pool of the available evidence in herbal interventions as opposed to ASEAN alone. Thus, the aim of the present study was to assess the reporting quality of RCTs of herbal interventions in ASEAN Plus Six Countries using the Elaborated CONSORT Statement.

Methods
Electronic searches for randomised controlled trials of herbal interventions from ASEAN Plus Six Countries were conducted in the following databases: PubMed, EMBASE, The Cochrane Library, and Allied and Complementary Medicine (AMED). Searches were performed from inception through October 2013, and were limited to studies specific to humans and randomised controlled trials with the use of the term ''random*''.
There were seven countries within the ASEAN Plus Six Groups where herbal medicines are listed as part of the National List of Essential Medicines or National Essential Drugs List [9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24]. These were China [10], Japan [12], South Korea [14], Lao PDR [18], Philippines [21], Thailand [23] and Vietnam [24]. However, the lists from China and Japan were written in Chinese and Japanese, respectively. Thus, the information was not readily available in English to us at the time of the study. In addition, we were unable to access the lists from South Korea and Lao PDR. The list from Philippines included herbal medicines from the several species including Cassia alata Linn. Vietnam has included thyme oil and ginko biloba in the list. Thailand has a long history of traditional and herbal medicine use. Over 70 herbal medicine products are listed under the National List of Essential Medicine (2011) [23]. This list was available to us at the time of the study. Of these herbal medicine products, we were interested in the products that were developed from single herbal species instead of the multiherb preparations as this enabled us to limit the scope of the present study. There were 20 herbal medicine products developed and manufactured from 20 herbal species that were included in the list. These 20 herbal species thus formed the basis for the search terms in the present study (Table S1). An electronic search strategy for a database (EMBASE) is given in Table S2.

Screening
One researcher (CP) performed the searches and screened the titles and abstracts of the identified studies. The following inclusion criteria were applied: (1) The studies were of herbal interventions of interest, single or combination preparations that included any of 20 species of herbs listed in the search terms (2) The studies were randomised controlled trials in human subjects ( Data extraction and assessment of quality of reporting Full text articles were retrieved; three reviewers (CP, RS, and PD) screened the content and performed data extraction using an electronic standardised extraction form (Data available on request). Discussions were held to resolve any disagreement or discrepancies.
For each randomised controlled trial, we extracted information on the reporting characteristics according to the 22-item Elaborated CONSORT statement of recommendations for reporting randomised controlled trials of herbal interventions [8]. An assessment of the quality of reporting was carried out whereby each CONSORT checklist item was assigned a yes or no response depending on whether the item was included in the study report under the recommended section of the report. Furthermore, additional items which were not part of the CONSORT checklist were evaluated to capture a more comprehensive picture of the study reports. These were clinical trial registration, the availability of full protocol, source of funding, authors' affiliations, ethics approval, acknowledgement, and disclosure of conflict of interest. Data were summarised using descriptive statistics with Excel version 2010.
We also compared the differences between ASEAN and Plus Six Countries to reveal information on the number of RCTs conducted, and the overall quality of reporting between the two Groups of Countries. Chi-Square tests were performed using SPSS Software (SPSS Inc. Released 2008. SPSS Statistics for Windows, Version 17.0. Chicago: SPSS Inc.). Fisher's Exact values were read when more than 20% of cells had expected values of less than 5.
The report of the present systematic review, where applicable, adhered to the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions [25].
were not herbal interventions of interest or were of countries of interest, or they were not published in English. Eighty-nine articles were selected for full text review, subsequently a further 18 studies were excluded because they were not of herbal interventions of interest, non-RCTs, unclear randomisation, the herbs were not being used as interventions or were published in non-English language. A total of 71 articles  were included (Figure 1).

General characteristics
The general characteristics of the 71 included RCT articles are presented in Table 1. The overall picture revealed that 30 (42.25%) RCTs were from ASEAN Countries, in contrast more than half (57.75%) were of the Plus Six origin. Thailand published the most RCTs of herbal interventions among the ASEAN Plus Six Countries (26 articles, 36.62%), which was followed by India (17 articles, 23.94%), and China (11 articles, 15.49%). Few studies were from Japan (5 articles, 7.04%) and South Korea (6 articles, 8.45%), while Philippines and Malaysia published the least having reported one article (1.41%) each.
From the scope of our interventions from 20 herbal species of interest, there were 12 species of herbs that were studied within the ASEAN Plus Six Countries. Zingiber officinale was the most frequently investigated herb (36 articles, 50.70%), and approximately one-tenth of the reported RCTs were of Curcuma longa L. (9 articles, 12.68%). The least investigated herbs were Clinacanthus nutans, Derris scandens Benth., Musa paradisiaca L., and Orthosiphon grandiflorus (1 article, 1.41% each).
With regard to the training or qualifications of first-authors, a total of 18 studies (25.35%) were from authors with Doctor of Medicine (MD) qualification, of which 4 had Doctor of Philosophy (PhD). Other studies were from a specialist in anaesthesiology (1.41%), a postgraduate resident (1.41%), and a nurse with a Bachelor of Science in Nursing (1.41%), etc. However, 46 studies (64.79%) did not clearly report the training or qualifications of first-authors, but they were affiliated with Medical College/ Medical University/Hospital (39.44%), Traditional Chinese Medical College/Hospital (5.63%), University (7.04%), or Research Centre/Institute (8.45%).
Most of the studies were published between 2001 and 2013 (94.37%), and only a few studies were published in the 1990s (5.63%). The highest publication records were seen in 2006 (16.90%) and 2011 (12.68%).
It was found that nausea and vomiting during pregnancy (7 articles, 9.86%), osteoarthritis of the knees (7 articles, 9.86%), and post-operative nausea and vomiting (6 articles, 8.45%) were the most studied clinical conditions. Other commonly studied conditions were type 2 diabetes mellitus, post-surgical pain, nausea and vomiting in cancer patients, and physical performance and cognitive function in the elderly (3 articles, 4.23% per each condition).

Quality of reporting
Overall CONSORT checklist items. The findings of the quality of reporting are represented in Table 2. Adherence to the recommended CONSORT checklist items for reporting of randomised controlled trials of herbal interventions among ASEAN Plus Six Countries ranged from 0% to 97.18%.
Fifteen CONSORT checklist items were reported, under the recommended sections, by most of the RCTs (more than 80%),      including: how participants were allocated to interventions, under the abstract section (e.g., ''random allocation'', ''randomised'', or ''randomly assigned''), statement of reasons for the trial with reference to the specific herbal medicinal product being used, the type of product used [e.g. raw (fresh or dry), extract], statistical methods used to compare groups for primary outcome(s), and discussion of the trial results in relation to trials of other available products.
Less than half of the RCTs reported the methods used to generate the random sequence allocation (item 8, 47.89%), and implement random allocation sequence (item 9 allocation concealment, 29.58%). Only one RCT completely reported the implementation of randomisation (item 10, 1.41%). In terms of blinding, approximately 31% RCTs clearly reported that they were double-blind studies with descriptions of who were blinded to group assignments (item 11). Around 27% RCTs reported diagrams of flow of participants through each stage (item 13). Albeit, 54.93% RCTs included drop-out reporting.
CONSORT checklist item 4. Under the CONSORT checklist item 4, where the report of precise details of the interventions intended for each group were recommended, more than half of the RCTs reported the part(s) of plant used to produce the product or extract (item 4B: characteristics of the herbal product, 53.52%), the type of product used (item 4B: characteristics of the herbal product, 81.69%), the content of all quantified herbal product constituents per dosage unit form (item 4C: dosage regimen and quantitative description, 80.28%), and the rationale for the type of control or placebo used (item 4E: placebo/control group, 95.77%).
However, only few studies presented the complete report of the method of authentication of raw material (item 4B: characteristics of the herbal product, 2.82%), the product's chemical fingerprinting and methods used (item 4D: qualitative testing, 4.23%), and the description of any special testing/purity testing (item 4D: qualitative testing, 5.63%). Less than a quarter of the RCTs (18.31%) reported information on standardisation of the herbal products (item 4D: qualitative testing).

ASEAN versus Plus Six Countries
Thirty RCTs (42.25%) were from ASEAN Countries, in contrast more than half (57.75%) were of the Plus Six Group. It can be clearly observed that studies from the Plus Six Group contributed larger proportions of the overall quality of reporting compared to studies from the ASEAN Countries, which could be directly related to the higher number of RCTs (41 studies versus 30 studies, respectively). This includes the method of authentication of raw material (item 4B) and product's chemical fingerprint and methods used (item 4D).
Results of Chi-Square Tests ( Table 2) showed that for most of the Elaborated CONSORT statement checklist items, there were no statistically significant differences between the quality of reporting of ASEAN and Plus Six Countries (P values.0.05). The checklist items that revealed statistically significant differences were 1) item 1 herbal medicinal product's Latin binomial (P = 0.010); 2) item 4A name of the manufacturer of the product (P = 0.006); and 3) Proprietary product name (brand name) or the exact name (P = 0.049).
It was noted that the checklist item 1 herbal medicinal product's Latin binomial was the only item that was reported more frequently in the RCTs of ASEAN Countries versus Plus Six Countries (P = 0.010).

Additional items
The results of the assessment of additional items are presented in Table 3. In general, ethics approval was reported by 74.65% of the RCTs, where almost all of the RCTs from ASEAN Countries (27 of 30 studies) provided ethics approval information, in comparison with the RCTs from Plus Six Countries, only 26 of 41 studies provided such information (P = 0.011). Reports on clinical trial registration and disclosure of conflict of interest were not common (7.04% and 23.94%, respectively). National and University/Institute were the most common sources of funding reported (14.08% for both sources), which was followed by funding from the industry (9.86%). Most of the authors (85.92%) were affiliated to universities or national institutes. Less than half of the studies (45.07%) acknowledged contributors other than the authors. No RCTs reported any information on the availability of full clinical trial protocol.

Discussion
Our study is the first to assess the reporting quality of RCTs of herbal interventions in ASEAN Plus Six Countries using the Elaborated CONSORT Statement. The study identified 71 RCTs of herbal interventions of interest, from the 20 herbal species listed in the NLEM (Thailand), which were conducted in the ASEAN Plus Six Countries. These RCTs were published since 1990. We generally found that none of the RCTs reported all of the CONSORT checklist items under the recommended sections. From the pool of herbal interventions of interest, more RCTs from Plus Six Countries were identified as opposed to those from ASEAN Countries. Thus explained why reports of RCTs from the Plus Six Group contributed larger proportions to the general quality of reporting. Few studies were from Australia, Japan and South Korea. This may be driven by the scope of the herbal species within the present study, which were limited to those from the list of Thailand's NLEM.
Importantly, we highlighted that the reporting quality was poor under the checklist items 4B (characteristics of the herbal product), 4D (qualitative testing), and 4F (descriptions of practitioners). Previous study of quality of reporting of RCTs of herbal medicine interventions by Gagnier who assessed 206 reports of RCTs [5] also commented that specific characteristics of the herbal intervention were not adequately reported. However, the result of our study on diagram of flow of participants through each stage (item 13) was different from that of Gagnier [5], where we found 19% of RCTs from ASEAN Plus Six Countries reported such item versus 75.2% from Gagnier. Furthermore, all important adverse events or side effects in each intervention group (item 19) were reported in 47% of RCTs in our study compared to 64.1% of Gagnier. Another point to note is that the interpretation of the results, taking into account study hypotheses, sources of potential bias or imprecision, and the dangers associated with multiplicity of analyses and outcomes, these items were assessed as one item in Gagnier study (reported by 68.5% of RCTs). In contrast, our study assessed these items separately and the results showed that 39% reported interpretation of the results in light of the product and dosage regimen used (item 20); and 32% reported sources of potential bias or imprecision. Moreover, our study assessed the reporting of item 21 that is where possible, discuss how the herbal product and dosage regimen used related to what is used in self-care and/or practice, which was reported by 50% of RCTs from ASEAN Plus Six Countries.
In the light of ASEAN integration where a single market is aimed for 2015, harmonisation of evaluation of traditional medicines and health supplements is the main focus of interest. Special attention should be directed to ensure safety, efficacy and quality of the herbal products. Specifically, the type of extraction solvent used where applicable (in the case of the product is an extract) in addition to the plant to plant extract ratio. This is of great value as it provides the readers with information on how much of the starting plant material was required to produce a specific amount of the finished extract [8]. Furthermore, RCTs should report method of authentication of raw material, product's constituents in terms of chemical profiling, especially special testing or purity testing as herbal medicines are often contaminated [97]. Moreover, standardisation of active chemical component(s) per dosage unit form should be reported, where applicable, to enable the audience to determine a specific effect in a clinical situation [5].
On the issue of randomisation and blinding, less than half of RCTs adequately reported information on method used to generate the random allocation sequence (47.89%), method for the implementation of random allocation sequence (29.58%), and blinding (30.99%). Other studies also found inadequate reporting of RCTs of herbal interventions [5], Traditional Chinese Medicine [98], and complementary and alternative medicine [4]. Such information is essential for readers to make judgment whether the results of the trials were subjected to bias.
Readers should be aware of the limitations within the present study. Only RCTs reported in the English language were included in the assessment. Hence, it may be considered as a language bias. In such manner, our findings can only be considered applicable to RCTs of herbal interventions reported in English. Although the selection of our databases may be considered as comprehensive, but this cannot be reciprocated for the selection or scope of our interventions of interest. Our selection was limited to those 20 herbal species listed in the NLEM of Thailand. Consequently, it may not represent the overall evidence base for ASEAN Plus Six Countries. This was clearly seen where there were more RCTs conducted in Thailand using those 20 herbal species compared to other countries within the Communities. Other countries in the ASEAN Plus Six Group may have conducted RCTs of other herbal species. Consequently, were not included in the present study as they were not part of the inclusion criteria. This may also have affected the number of the included studies, and the overall picture of the reporting quality. Nevertheless, our study represents a starting point for the assessment of RCT reporting quality of herbal interventions from ASEAN Plus Six Group.

Conclusion
Traditional Medicine is of great importance and plays an integrative role in the ASEAN Communities. The present study highlights the need to improve reporting quality of RCTs of herbal interventions across ASEAN Plus Six Group. Efforts to support research associated with the products of herbal origins should aim to improve the quality of RCTs, including the complete and adequate reporting thereof.

Supporting Information
Checklist S1 PRISMA 2009 Checklist for systematic review. (DOC)