Survey of basic medical researchers on the awareness of animal experimental designs and reporting standards in China

Objective To investigate the awareness and use of the Systematic Review Center for Laboratory Animal Experimentation’s (SYRCLE) risk-of-bias tool, the Animal Research: Reporting of In Vivo Experiments (ARRIVE) reporting guidelines, and Gold Standard Publication Checklist (GSPC) in China in basic medical researchers of animal experimental studies. Methods A national questionnaire-based survey targeting basic medical researchers was carried in China to investigate the basic information and awareness of SYRCLE’s risk of bias tool, ARRIVE guidelines, GSPC, and animal experimental bias risk control factors. The EpiData3.1 software was used for data entry, and Microsoft Excel 2013 was used for statistical analysis in this study. The number of cases (n) and percentage (%) of classified information were statistically described, and the comparison between groups (i.e., current students vs. research staff) was performed using chi-square test. Results A total of 298 questionnaires were distributed, and 272 responses were received, which included 266 valid questionnaires (from 118 current students and 148 research staff). Among the 266 survey participants, only 15.8% was aware of the SYRCLE’s risk of bias tool, with significant difference between the two groups (P = 0.003), and the awareness rates of ARRIVE guidelines and GSPC were only 9.4% and 9.0%, respectively; 58.6% survey participants believed that the reports of animal experimental studies in Chinese literature were inadequate, with significant difference between the two groups (P = 0.004). In addition, only approximately 1/3 of the survey participants had read systematic reviews and meta-analysis reports of animal experimental studies; only 16/266 (6.0%) had carried out/participated in and 11/266 (4.1%) had published systematic reviews/meta-analysis of animal experimental studies. Conclusions The awareness and use rates of SYRCLE’s risk-of-bias tool, the ARRIVE guidelines, and the GSPC were low among Chinese basic medical researchers. Therefore, specific measures are necessary to promote and popularize these standards and specifications and to introduce these standards into guidelines of Chinese domestic journals as soon as possible to raise awareness and increase use rates of researchers and journal editors, thereby improving the quality of animal experimental methods and reports.


Introduction
As an important bridge between basic medical research and clinical trials, animal experimental studies are important to validate the safety and efficacy of the interventions and to determine whether new interventions can be applied to clinical trials [1,2]. A large number of published animal studies have varying degrees of defects in their experimental designs, implementation, and reporting methods, which seriously affects the authenticity and reliability of these animal studies, and poses difficulties in replicating results [3][4][5][6][7].
To address this problem, Animal Research: Reporting In Vivo Experiment (ARRIVE) guidelines [8] and the Gold Standard Publication Checklist (GSPC) [9] were published to standardize animal experimental reports, to ensure sufficient evaluation and utilization of animal experimental data, and to facilitate integrity and transparency during the review processes of basic medical research [10,11]. Following the publication of these standards, to design more rigorous animal experimental designs and to effectively control risk of bias, Hooijmans and other researchers from the Systematic Review Centre for Laboratory Animal Experimentation (SYRCLE) in the Netherlands used Cochrane's risk-of-bias tool (as recommended by the Cochrane Handbook [12]) to study, draft, and develop SYRCLE's risk-of-bias tool for animal experimental studies, which was published in 2014 [13]. These guidelines have important implications for the scientific design, rigorous implementation, and regulatory reporting of animal experimental studies. Although these guidelines were introduced in China several years ago [10,11], a previous study [14] demonstrated that the editorial staff of Chinese domestic journals that publish animal experimental studies have a limited awareness of this information and low implementation rates of the ARRIVE guidelines and GSPC. In addition to the journal editors, basic medical researchers have significant influence on the degree of bias in the design and implementation of animal experimental studies and on the adequacy of the reported results. Although a study by Fang et al. [15] in China conducted a relevant survey with animal experimental researchers in Lanzhou City, Gansu Province, China, the survey was limited to the awareness of the ARRIVE guidelines and GSPC.
Therefore, this study conducted a questionnaire-based survey to expand the scope of survey respondents and contents to comprehensively investigate awareness of the design and implementation of animal experimental methods as well as reporting standards and their use. This study identified problems and promoted the popularity of these guidelines to basic medical researchers to improve the quality of animal experimental studies.

Survey respondents
Survey respondents include current students (i.e., master and doctoral students in the graduate school) and research staff members (e.g., university faculties, researchers, experimental technicians, and clinicians) in the field of basic medical research. Convenience sampling was used and respondents were from six major geographic areas of China, including 23 medical institutes and affiliated hospitals in Northwest China, Southwest China, South China, North China, East China, and Northeast China.(S1 File)

Survey methods
With reference to relevant literature and combined with the details of SYRCLE's risk-of-bias tool, the ARRIVE guidelines, and the GSPC, we designed the questionnaires in the form of single-answer and multiple-choice questions. The questionnaires (S2 File) were distributed through e-mail, the postal service, and on-site delivery and were collected and gathered by specialized investigators(XU Jia-ke, WU Wen-jing, LIU Hong-yan, KOU Cheng-kun) to ensure authenticity and legitimacy of the survey results. The survey was conducted from June to August 2016.

Survey contents
The survey contents included (1) general information about survey respondents (e.g., academic degree and occupation), time had spent in basic medical teaching or research, whether had participated or directed any projects on animal experimental studies, number of articles related to animal experimental studies read per month, and means to access relevant research progress; (2) awareness of the items of SYRCLE's risk-of-bias tool; (3) status of relevant published reports on animal experimental studies; (4) whether had written or published papers on animal experimental studies, awareness and source of the ARRIVE guidelines and GSPC; and (5) awareness of systematic review and meta-analysis of animal experimental studies.

Ethical review
The Ethical Committee of School of Basic Medical Sciences, Lanzhou University, Lanzhou, Gansu, China, approved this survey. Given that this was a survey of opinion, with no risk of harm to respondents, we obtained informed consent verbally.

Statistical analysis
For this study, EpiData 3.1 software was used for data entry, and Microsoft Excel 2013 software was used for statistical analysis. The count data were described statistically using the number of cases (n) and percentage (%). Comparison of classified information (nongrade data) between current students and research staff was performed using chi-square test. The test level α was set to 0.05, and P < 0.05 was considered to be a statistically significant difference. Table 1) 33/266) and clinicians (7.5%, 20/266). Most of the survey respondents held master's degrees (55.6%, 148/266) or doctoral degrees (25.2%, 67/266).

General information about survey respondents (see
More than half of the 266 survey respondents (57.1%, 152/266) spent less than five years in basic medical teaching or research; most of the survey respondents (89.8%, 239/266) developed or had participated in animal experimental studies; 74.4% (198/266) of the survey respondents were the main researchers who had participated in animal experimental studies.
Nearly half of the 266 survey respondents (47.7%, 127/266) read less than five articles pertaining to animal experimental studies per month. The vast majority of respondents accessed relevant research updates from medical databases (90.2%, 240/266), followed by respondents  Table 2)

Awareness and knowledge of SYRCLE's risk-of-bias tool and bias risk control factors (see
Of the 266 survey respondents, only 15.8% (42/266) had heard of SYRCLE's risk-of-bias tool, with a statistically significant difference between groups (P = 0.003). Only 3.0% (8/287) understood and were familiar with SYRCLE's risk-of-bias tool (without significant difference between groups, P = 0.972). Among the items included in SYRCLE's risk-of-bias tool, most survey respondents considered the first item ( Table 3)

Investigation of existing literature in animal experiments (see
Of the 266 survey respondents, 58.6% (156/266) considered existing literature on animal experimental studies to be inadequate (significant difference between groups, P = 0.004).
More than 50% of the survey respondents believed that existing literature on animal experimental studies was inadequate in the following aspects: • Background section: "Explain how and why the animal species and model being used can address the scientific objectives and, where appropriate, the study's relevance to human biology" (67.9%, 106/266; significant difference between groups, P = 0.005).
Lower than 50% and More than 30% of the survey respondents considered existing literature on animal experimental studies to be inadequate in the following aspects: • Background section: "Interpretation of study objective(s), contents, experimental methods, and basic principles" (32.7%, 51/266; no significant difference between groups, P = 0.906).

Awareness of the ARRIVE guidelines and GSPC(see
Among the 266 survey respondents, 53.8% (143/266) and 45.1% (120/266) of the survey respondents participated in animal experimental studies and published animal experimental papers, respectively (significant difference between groups, P < 0.05). When preparing and reporting animal experimental results, the majority of survey respondents (82.0%, 218/266) primarily referred to publications of similar studies, followed by reference to the editorial requirements of the target journals (36.5%, 97/266) and the target journal's guidelines (given in the introduction to authors, 32.3%, 86/266). Of the 266 survey respondents, few respondents had heard of the ARRIVE guidelines (9.4%, 25/266) and GSPC (9.0%, 24/266), and very few respondents understood or were familiar with the ARRIVE guidelines (3.0%, 8/266) and GSPC (3.4%, 9/266). Most of the survey respondents (82.3%, 219/266), however, considered the necessity of studying animal experimental checklists to standardize the reporting of results and manuscript preparation.
3.5. Awareness of systematic reviews and meta-analysis of animal experimental studies (see Table 5) Although most of the respondents (77.4%, 206/266) had heard of systematic reviews and metaanalysis methods, of the 266 survey respondents, only about a third (31.2%, 83/266) had read about these published systematic reviews or meta-analysis of animal experimental studies (see Table 2). Very few respondents had conducted (6.0%, 16/266) or published (4.1%, 11/266) systematic reviews or meta-analysis of animal experimental studies. Most of the survey respondents, however, believed that systematic review or meta-analysis would be an effective way to increase the value of animal experimental studies to clinical research.

Discussion
Animal experimental studies not only provide preliminary validation of the safety and efficacy of interventions but also provide scientific evidence to determine whether new interventions can enter the clinical research phase. This validation must be based on scientific design, rigorous implementation, and reports with standardized results. This study's survey of animal experimental methods showed that most survey respondents believed that "randomization (85.7%)," "baseline characteristics (78.9%)," and "incomplete data reporting (81.6%)" play a very important role in the design of animal experimental studies. Less than 50% of the survey respondents, however, believed that the implementation of "allocation concealment," "animal randomization," and "blindness" (e.g., focusing on the animal breeders, researchers, and personnel performed outcome measurement) or measures to avoid "selective outcome reporting" were very important in animal experimental studies. Allocation concealment hides the generated random sequence number to ensure that participants do not know the random sequence number. This measure is used to control and reduce selection bias along with animal randomization [16]. Some international studies have shown that implementing strict randomization to generate random sequence numbers but not applying allocation concealment results in an exaggeration of treatment efficacy by 30-41% [16]. Similar to clinical studies, implementation of blindness in animal experimental studies is essential, especially for the subjective outcome measurements of some indicators to effectively avoid and reduce the bias in implementation or measurement [13,17]. In contrast to clinical studies, "animal randomization" and "evaluation of random outcomes" are very important to control for risk of laboratory bias in animal experimental studies. Results of this study, however, showed that only 30% of the survey respondents believed "animal randomization" to be very important, and only 50% of the survey respondents considered the "evaluation of random outcomes" to be very important. Regarding different housing conditions of the experimental animals, different light intensity and temperatures of the animal facilities have important impacts on the results of these animal studies. For example, numerous studies have shown that animal cages placed at the higher position of the rack are exposed to four times the light intensity compared with the animal cages placed at the bottom of the rack [13,18]. Slight alterations of light intensity affect and change the reproduction and behavior of the animals [13,[19][20][21]. In addition, temperature variations inside animal cages result from their placement at different levels of the rack (cages at the top of the rack are typically 5˚C higher than cages at the bottom of the rack) [13,18,22,23]. Conversely, if animals are not randomized, researchers may anticipate the behaviors of each group of animals, resulting in implementation bias. Moreover, in animal experimental studies, random selection of experimental animals is very important for the evaluation and measurement of study outcomes. Because most living beings have circadian rhythms, such as lipid (compound) metabolism, neurotransmitter levels, and pharmacokinetics, which affect the variations of cycle and circadian rhythms of animals [13,[24][25][26], measurement bias may be introduced when sample evaluations and measurements are conducted only in certain periods of time without applying randomization during the measurement. SYRCLE's risk-of-bias tool not only helps researchers to standardize the design of animal experimental studies but also provides a specific set of tools to assess authenticity in these experiments. In this study, the survey of experimental methods used in animal studies was based on this assessment tool. In 2014, our research team had introduced and described SYR-CLE's risk-of-bias tool for animal experimental studies in detail [11], but the results of this current survey show that only 15.8% of the survey respondents had heard of SYRCLE, and only 3% of the survey respondents understood and were familiar with this assessment tool. This low level of awareness might explain the overall low quality of animal experimental studies in China. On the basis of these findings, it was necessary to further increase the promotion of SYRCLE's risk-of-bias tool in China to encourage basic medical researchers to design animal experimental studies more scientifically and rigorously to improve the authenticity of these experiments.
In this study, a survey of reporting standards of animal experimental studies showed that less than 10% of the survey respondents had heard of the ARRIVE guidelines and GSPC, and less than 5% of the survey respondents understood and were familiar with these two reporting standards. When writing manuscripts detailing animal experimental design, most of the respondents referred primarily to published literature of relevant studies (82.0%, 218/266), the requirements of the target journals (36.5%, 97/266), and the introduction to authors provided by the target journal's guidelines (32.3%, 86/266). A previous study [14], however, has shown that the 240 domestic journals in China that have published animal experimental studies do not mention the ARRIVE guidelines or GSPC in their journal guidelines. In addition, the editorial staff of the relevant journals had very low awareness of these reporting standards. This lack of awareness may explain the inadequacy of reporting animal experimental results in the majority of studies conducted in China, which was confirmed in this study. When the 266 survey respondents read the animal experimental literature, more than 50% believed that the existing literature on animal experimental studies was inadequate, primarily in the methods and results sections. Since these reporting standards were issued, the ARRIVE guidelines and GSPC have been included in the introduction to authors provided by 317 international journals [27] and were endorsed and recommended by International Committee of Medical Journal Editors and EQUATE Network [28]. A study [29] showed that the use of ARRIVE guidelines and GSPC can improve the quality of reports on animal experimental studies. The ARRIVE guidelines and GSPC provide important guidance about manuscript preparation for animal experimental studies, which will enable readers to accurately and clearly understand the content of these manuscripts. Moreover, these guidelines provide important information required for the replication of the animal experiments [8] to ensure that the information about these animal experimental studies will be assessed adequately and utilized to facilitate integrity and transparency of the basic medical research review process and to avoid wasting medical resources [30]. Therefore, we highly recommend introducing the ARRIVE guidelines and GSPC to Chinese domestic journals that publish animal experimental studies and related studies to improve the awareness and actual use rate of these reporting standards.
A limitation of our study was used convenience sampling rather than random sampling, even though convenience sampling has a high rate of feedback. In addition, participants in our survey study were from six major geographic areas of China, but the number of participants relative to the estimated size of the target population did not evaluated which may affected the outcome of our survey study.

Conclusions
Basic medical researchers in China had low awareness and use of SYRCLE's risk-of-bias tool, the ARRIVE guidelines, and GSPC in animal experimental studies conducted in China. Therefore, we proposed taking specific measures to promote and popularize these guidelines and to introduce these guidelines in Chinese domestic journals as soon as possible. In so doing, we hope to improve awareness and actual use rates of these guidelines by basic medical researchers and journal editors, thereby improving the quality of animal experimental methods and reporting standards.
Supporting information S1 File. Name of medical institutes and affiliated hospital in our survey study.