Adherence to ARRIVE Guidelines in Chinese Journal Reports on Neoplasms in Animals

Background The Animals in Research: Reporting In Vivo Experiments (ARRIVE) guidelines were published in 2010 with the aim of improving the quality of studies involving animals. However, how well Chinese studies involving animal neoplasms adhere to these guidelines has not been assessed. Objective To evaluate the reporting quality of such experiments published between 2010 and 2012 in Chinese journals with support from the National Natural Science Foundation of China. Methods We searched the Chinese Science Citation and Chinese Journal Full-Text Databases for articles published between 2010 and 2012 involving neoplasms in animals. The data were extracted into pre-prepared forms. Reporting quality was assessed using the ARRIVE checklist—39 items plus information on blinding. Results Three hundred and ninety-six animal studies were included in the analysis: 127 studies published in 2010, 140 studies published in 2011, and 129 studies published in 2012. The range of ARRIVE score is from 12 to 27 with a maximum possible score of 40. Studies published in 2012 (P = 0.012), 2011 (P = 0.015), 2010, July~Dec (P<0.017) had a significantly larger ARRIVE checklist score than those published in Jan.~June, 2010, respectively. Conclusions Experiments involving neoplasms in animals published in Chinese journals generally have not comprehensively reported the information recommended by the ARRIVE guidelines. We strongly recommend that researchers conducting such studies report this information.


Objective
To evaluate the reporting quality of such experiments published between 2010 and 2012 in Chinese journals with support from the National Natural Science Foundation of China.

Methods
We searched the Chinese Science Citation and Chinese Journal Full-Text Databases for articles published between 2010 and 2012 involving neoplasms in animals. The data were extracted into pre-prepared forms. Reporting quality was assessed using the ARRIVE checklist-39 items plus information on blinding.

Introduction
Animals are often used during biomedical research studies, but such studies are often controversial because there are many differences between animals and humans. The aims of preclinical animal experiments are to perform preliminary safety and efficacy validations of the (new) intervention under study, with the results determining whether the intervention should be further assessed in clinical studies and trials. Every year, many animal experiments, including those supported by foundations and other organizations, are published in a variety of journals. In China, the National Natural Science Foundation of China (NSFC) is one of the major funding sources for basic science research.
Although random allocation and blinding are common in clinical trials, they are not common in animal experiments that precede clinical trials. One study [1] assessed 290 published animal experiments and reported that 32.41% (94/290) used randomization and 9.66% (28/ 290) used blinding. Non-randomized, non-blinded animal studies are more likely to report a difference between study groups than animal studies that use these methods [1]. Experimental design, statistical analysis, and reporting issues have also been found for animal studies [2,3].
The Animals in Research: Reporting In Vivo Experiments (ARRIVE) guidelines published in 2010 were developed to improve the transparency and accuracy of bioscience research reporting. The guidelines detail the minimum information that should be reported when using animals in a research study and include a 20-item checklist [4]. In 2011, the ARRIVE guidelines were introduced in China [5]. To date, no study has assessed whether animal experiments of neoplasms published in Chinese journals adhere to these guidelines.
The aim of this study was to determine the reporting quality of experiments, specifically those involving neoplasms in animals, which were published in Chinese journals between 2010 and 2012 and were supported by the NSFC.

Inclusion/exclusion criteria
The targeted studies were those involving neoplasms in living rodents, including rats, mice, nude mice, and guinea pigs (Mammalia: Rodentia), published between 2010 and 2012 in Chinese journals, and supported by the NSFC.
We excluded studies on animals that were dead before the start of the experiment, when the studies were invitro, when no intervention was performed and no control group was included, when the studies focused on precancerosis, or when the studies were part of an academic dissertation or review article.
July 2012 to December 2012. The main search terms were "neoplasm," "animal experiment, in vivo experiment, basic research," and "National Natural Science Foundation." The search strategy is presented in S1 File.

Screening
The titles and abstracts were independently screened by at least two reviewers (Xingxing Zhao, Yuefeng Mai, Xinxin Li, and/or Wanting Sun). Then, the full texts of potentially suitable articles were retrieved based on the inclusion and exclusion criteria. Disagreements concerning the suitability of an article were resolved by group discussions.

Data extraction
Extraction into a pilot-tested standardized data form based on the ARRIVE guidelines was performed independently by at least two reviewers (Xingxing Zhao, Yuefeng Mai, Xinxin Li, Jin Wang, Lili Chen, Jing Mu, Gengxue Jin, and/or Hongping Gou). Inconsistencies were subsequently resolved by discussion among two or more reviewers, or the principal investigator Yali Liu made the final decision. The form consisted of two sections: (1) general characteristics (publication time, role of first author, the condition of interest, the allocation method of the included animals, and the number of funding organizations), (2) the ARRIVE information (39 items), and information about blinding that was separated from the checklist as item 6f and written as "Describe the information: If done, describe who was blinded (for example, outcome assessors) and how", i.e., for a total of 40 scored items. Each item was assessed as "yes" (the item was described in the study) or "no" (the item was not described in the study).

Data analysis
The data were summarized using Microsoft Excel (Version 2007; http://office.microsoft.com/ zh-cn) and SPSS software (Version 21.0; http://www.spss.com). For continuous variables (for example the ARRIVE checklist scores), we expressed results as median and quartile (P 25 , P 45 ) and comprised them for Kruskal-Wallis H Test. We use the traditional 0.05 definition of significance. The Bonferroni method was used for correcting for multiple comparisons. We mainly focus on a contrastive analysis of animal experiments before and after the publication of ARRIVE guidelines. We made 3 comparisons, then the new threshold is 0.017 (0.05/3).For categorical variables (for example the reported rate of ARRIVE checklist items), we expressed result as frequently and comprised them for the chi-square test.

The literature search
We initially identified 2846 studies. Of these, 1889 were excluded given their title or abstract, and a further 520 were excluded after an assessment of their full texts because they did not meet the inclusion criteria. The full texts of an additional 41 articles were not available. The remaining 396 studies met the inclusion criteria and were included in the analysis (Fig 1 and  S2 File).

General Characteristics of the Included Studies
The characteristics of the included studies are shown in Table 1. Of these studies, 126 (31.82%), 141 (35.61%), and 129 (32.58%) were published in 2010, 2011, and 2012, respectively. There were 77 studies which were published the release of the before ARRIVE guideline. For slightly more than half of the studies (219/396; 55.30%), the first author was affiliated with a hospital.
The most commonly studied conditions were liver and lung cancers. The majority of the studies randomly assigned animals to the experimental or control group, but only 26.77% (106/396) described the randomization method in detail. Eleven studies used a random number table to perform randomization. Other randomization methods are described in Table 2.A total of 159 studies (40.15%) were supported by only the NSFC. The other 237 studies (59.85%) were supported by up to five sources.

Conformity with ARRIVE Guidelines
The ARRIVE checklist scores are shown in Table 2    The least frequently reported items (reported in 30% of the studies) were items 3b, 5, 6d, 6f, 7c, 7d, 9b, 9c, 10b, 10c, 11a, 13b, 13c, 14, 15a, 15b, 17a, 17b, 18b, 18c, and 19. These items correspond to information on the scientific background, ethical statements, experimental procedures, sample size, statistical methods, baseline data, numbers of animals analyzed, adverse events, and interpretation/scientific implications. No study provided a time-line diagram or flow chart, information on sample size, information on how the experimental protocol reduced adverse events, or experimental methods or findings for the replacement, refinement, or

Discussion
Over the last decade, numerous studies have examined the quality of healthcare reporting by assessing the compliance of randomized controlled trials [6],observational studies [7], and systematic reviews [8] with various assessment instruments. The CONSORT statement published in 1996 [9], the ARRIVE guidelines [4], and the Gold Standard Publication Checklist (GSPC) [10] published in 2010 represent substantial improvements in methods used for animal studies. Before these guidelines, insufficient reporting occurred in many animal experiments, e.g., a lack of randomization and/or blinding [11][12][13]. The reporting quality of animal experiments in periodontology, e.g., implant dentistry, published after 2010 has been assessed using the ARRIVE and modified ARRIVE guidelines [2,3,[14][15][16][17].
In the present study, we focused on 396 reports involving neoplasms in rodents published in Chinese journals between 2010 and 2012. To our knowledge, this is the first assessment of this type of study published in Chinese journals and supported by the NSFC. Although we did not perform an intervention systematic review, we tried to conform to the preferred reporting items for the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA statement) checklist [8]. In our study, we found that the completeness of these studies regarding the ARRIVE guidelines was suboptimal. The mean ARRIVE checklist score was only 19.48 out of 40, and many studies failed to report important information. Furthermore, half the items in the ARRIVE checklist were reported by <50% of studies. Items 3b, 5, 6d, 6f, 7c, 7d, 9b, 9c, 10b, 10c, 11a, 13b, 13c, 14, 15a, 15b, 17a, 17b, 18b, 18c, and 19 were particularly poorly reported.
All studies included in this analysis were supported by at least one funding organization. Only two studies reported that no conflict of interest existed, and the other studies did not provide a declaration of interests. Accurate reporting of conflicts of interest is very important to enable readers to judge the risk of publication bias.
We found that few studies (1.77%) provided a statement of ethics or provided information on animal welfare. Researchers need to pay close attention to study design, data collection, reporting, and the welfare of the animals to take effective measures to alleviate animal suffering. The ARRIVE guidelines recommend providing a time-line diagram or flow chart of the study design, but neither was part of any study. Similarly, no study described how the sample size was calculated, and therefore it was unclear if the sample sizes were adequate. To minimize the effect of random error on the results, studies should include independent repetitions of each experiment. Although many studies reported the number of repeated measurements, many failed to describe the number of independent replications for each experiment.
As with clinical trials, animal experiments should report if the subjects were randomized into groups and also describe the randomization method in detail to help readers assess the risk of selective bias. Furthermore, it is important to blind outcome assessors and data analysts to the group assignment, thereby reducing the risk of measurement bias for some subjective outcomes. However, blinding was used in only one study.
Although all studies reported the experimental outcomes, many did not define primary and secondary outcomes. Information concerning statistical methods was often incomplete, e.g., the unit of analysis for each dataset was not specified, or the rationale for selecting the statistical approach was not provided.
Only one studies (0.25%) provided the relevant characteristics and health status of the animals, e.g., the strain, sex, age, and weight of the experimental and control animals, which are important factors that allow the reader to judge if the groups were or were not balanced at baseline.
Insufficient information was often provided on the number of animals studied, and no descriptions or explanations were provided concerning animal death or loss to follow-up or incomplete outcome data, making it difficult for readers to evaluate the risk of attrition bias. In addition, although all studies reported the efficacy of the treatment, only 41 (10.35%) reported adverse events, and none described the approach(s) used to reduce adverse events.
Ideally, the discussion section of such studies should address how the results translate to other systems or species and the feasibility of testing the treatment in clinical trials. When animals are used in research, the researcher has the responsibility to work toward the replacement, refinement, and reduction of animal use, i.e., to improve the use value of the animals, reduce the number of animals used, and increase the use of alternative methods. Unfortunately, none of the studies reported information about these issues.
All experimental research has limitations; however, many of the studies did not address this point.
Analysis reveals that the reporting quality of animal experiments reports shows a rising tendency after the first publication of the ARRIVE in June, 2010.However, the quality of reporting was still poor, and future studies should be more transparent and accurately reported. We believe that reporting guidelines, including those of ARRIVE and GSPC, have not been widely used by Chinese researchers who use animals, as our results indicate that neither the ARRIVE nor the GSPC guidelines were used by such researchers from 2010 July to 2012. The ARRIVE and GSPC guidelines outline the minimum information that should be provided when reporting an animal experiment(s). These guidelines should be used when designing, performing, reporting, reviewing, and publishing an animal study. They will also be of use to funding organizations, especially large funding bodies, which have a responsibility to provide financial assistance and to strictly evaluate the quality of the research that is funded [18]. Good reporting allows readers to fully understand the methods and processes involved and to assess the reliability and validity of the findings.
There are several limitations to our study. First, our analyses were limited to studies on rodents, and studies on other animals were excluded. Second, we only included animal experiments involving neoplasms that were published in Chinese journals between 2010 and 2012. Third, our scoring criteria (yes or no) did not allow for partial information. Fourth, we used unweighted scores for the ARRIVE checklist items, although this may not be a valid approach. Finally, in general, the reporting quality of experiments involving neoplasms in animals in Chinese journals may be worse than what we found because all the studies included in our analyses were supported by the NSFC.