Figures
Abstract
Purpose
To explore the readability and content integrity of informed consent forms (ICFs) used in China and to compare the quality of Chinese local ICFs with that of international ICFs.
Methods
The length, readability and content of 155 consent documents from phase II-IV drug clinical trials from the Third Xiangya Hospital Ethics Committee from November 2009 to January 2015 were evaluated. Reading difficulty was tested using a readability formula adapted for the Chinese language. An ICF checklist containing 27 required elements was successfully constructed to evaluate content integrity. The description of alternatives to participation was assessed. The quality of ICFs from different sponsorships were also compared.
Results
Among the 155 evaluable trials, the ICFs had a median length of 5286 words, corresponding to 7 pages. The median readability score was 4.31 (4.02–4.41), with 63.9% at the 2nd level and 36.1% at the 3rd level. Five of the 27 elements were frequently neglected. The average score for the description of alternatives to participation was 1.06, and 27.7% of the ICFs did not mention any alternatives. Compared with Chinese local ICFs, international ICFs were longer, more readable and contained more of the required elements (P < 0.05).
Conclusion
The ICFs used in China were difficult to read for most participants. These forms had poor description of alternatives to participation, and failed to provide a high degree of information disclosure, including an explanation of informed consent, follow-up processing of the data/sample, inclusion/exclusion criteria, double blinding, and unpredictable risks. International ICFs had better readability and content integrity than Chinese local ICFs. More efforts should thus be made to improve the quality of consent documents in China.
Citation: Wen G, Liu X, Huang L, Shu J, Xu N, Chen R, et al. (2016) Readability and Content Assessment of Informed Consent Forms for Phase II-IV Clinical Trials in China. PLoS ONE 11(10): e0164251. https://doi.org/10.1371/journal.pone.0164251
Editor: Robert K. Hills, Cardiff University, UNITED KINGDOM
Received: June 20, 2016; Accepted: September 20, 2016; Published: October 4, 2016
Copyright: © 2016 Wen et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper.
Funding: Hong Yuan received the National Natural Science Foundation of China (81470535), Gaiyan Wen received the Fundamental Research Funds for the Central Universities of Central South University (2015zzts320), and Yao Lu received the Fundamental Research Funds for the Central Universities of Central South University (2016zzts153); Hong Yuan and Yao Lu were employed in the study design, Gaiyan Wen were employed in the manuscript preparation.
Competing interests: The authors have declared that no competing interests exist.
Introduction
With the trend of international globalization, China is playing an important role in conducting clinical trials [1, 2]. More than 800 types of new drugs involving 500,000 Chinese subjects have been evaluated in clinical trials every year. As a cornerstone of ethical healthcare research, an informed consent form (ICF) is a requirement for drug clinical trials to provide protection for participants in these trials [3]. Participants can obtain useful information, weigh benefits and risks, and make an informed and voluntary decision to participate from ICFs. Ideally, a valid ICF must provide all necessary information in a way that is easily understood [4].
Although researchers have emphasized the importance of the standardization and perfection of ICFs, many incomplete or inaccurate facets remain. Numerous studies have shown that most potential participants fail to comprehend ICF content [5–9]. Participants commonly misinterpret the purpose of clinical trials, their benefits and risks, side effects, and the ability to withdraw from participation [6–9]. These deficiencies lead to poor comprehension of ICF information and potentially threaten the safety of subjects. Also, ICFs should be written in simple, easy-to-read language to ensure that participants can fully understand all the information in ICFs [10]. However, as a matter of fact, some researchers have not fully accounted for the age, education level and mental ability of subjects or their representatives [11], which may increase the likelihood of regretting their participation or withdrawing from the study [12].
Some previous studies have evaluated the content and readability of ICFs. In 2003, Paasche-Orlow conducted a large-scale, cross-sectional study using the Flesch-Kincaid readability scale to measure the readability of ICFs from institutional review boards (IRBs) [13]. The results showed that IRBs fall short of their own readability standards and need language improvements. Three separate studies [14–16] developed surveys of the content and readability of ICFs for clinical oncology trials and found that consent documents were too lengthy and complex for participants to read. France, Croatia and New Zealand were facing a similar problem in the poor quality of ICFs [11, 17, 18]. As China is undertaking an increasing number of clinical trials, it is of great importance to clarify the quality of ICFs in China. Notably, ICFs from phase I clinical trials were under the intense scrutiny of IRBs and had higher readability scores [16], as these trials primarily target healthy volunteers and aim to evaluate drug risks. By contrast, the quality of ICFs for phase II-IV clinical trials involving diverse participants is inconsistent.
Our study aimed to explore the readability and content integrity of ICFs for phase II-IV clinical trials in China for the first time and to compare the quality of Chinese local ICFs with that of international ICFs, aiming to provide insights into the deficiencies and misunderstanding of consent documents and provide a sound basis for the improvement of ICFs in China.
Materials and Methods
Selection criteria for ICFs
ICFs for phase II-IV drug clinical trials approved by the Third Xiangya Hospital Ethics Committee in Changsha, China, from November 2009 to January 2015 were considered for inclusion in this study. ICFs for phase I clinical trials, bioequivalence researches, pharmacokinetic studies and tolerance studies were excluded. Moreover, this study excluded ICFs for medical apparatuses and techniques and those mainly involving questionnaires. Scanned materials were also excluded because of the unavailability of word counts and readability scores. To avoid duplicative reviews, only the original version of each ICF that was approved by the ethics committee was evaluated.
Approach to data collection
The following items were collected, including details regarding the phase of the study, administration route, drug classification and sponsor. These documents were classified by sponsor as Chinese local ICFs and international ICFs. The sponsors of international ICFs were multinational pharmaceutical companies, including Bayer HealthCare AG, AstraZeneca Pharmaceutical Co. Ltd, Beijing Novartis Pharma Ltd, GlaxoSmithKline Plc and several Japanese pharmaceutical companies, and the sponsors of Chinese local ICFs were Chinese local pharmaceutical companies.
Readability assessment
All of the materials evaluated were written in Chinese and available in electronic form. The readability analysis was based on an assessment of the text length, font size, number of pages, presence of a flow chart, use of appropriate sub-headings, and readability score. The number of words, font size and number of pages were determined by Microsoft Word. The presence of a flow chart was checked manually. For the English language, readability formulas such as the Flesch Reading Ease score (FRE) and the Gunning Fog Index (GFI) are based on word complexity and sentence length, but these formulas cannot be directly applied to Chinese because of special characteristics of the language [19, 20]. The readability formula adapted for the Chinese language in 1974 by Yang was used in our study to assess the reading difficulty of the material, which correlates well with comprehension tests [21]. The formula was as follows: Y = 3.5921 + 0.8826X1−0.0179X2, where Y is a comprehension score, X1 is the number of words with a stroke number less than 10 divided by the total number of words, and X2 is the number of special characters divided by the total number of words. As no computer software with a Chinese readability formula was available, all calculations were performed manually. According to Fry’s methods [22], three paragraphs (each paragraph included 100 words) were randomly selected near the beginning, middle, and end of the document. The stroke number of every word and the number of special characters in the three paragraphs were then counted. Subsequently, X1 was calculated by dividing the number of words with a stroke number less than 10 by 300. X2 was calculated by dividing the number of special characters by 300. The comprehension score was evaluated from level 1 to 5, with higher values indicating better readability. The readability levels are reported on a standardized normal distribution of the difficulty of the material but are not tied to the concept of educational level. According to a 2012 Chinese residents' health literacy monitoring report [23], only 8.80% of the residents had an adequate level of health information literacy in China. The overall level of health literacy and education is relatively lower in China than in well-developed countries. For our analysis, the “2nd easy” level is generally associated with a secondary school or higher education level, and the “3rd median” level is associated with a vocational or high school level.
Content assessment
A content evaluation tool for ICFs was designed according to the World Medical Association Declaration of Helsinki, the international ethical guidelines for biomedical research involving human subjects, the International Conference on Harmonization Clinical Practices Guide (ICH GCP), other regulations and specifications, and the references [24–26]. Composed of 27 items, this tool consists of four sections, including general items, rights of subjects, scientific aspects, and ethical aspects. Two researchers rated all items independently for each ICF, and inconsistent findings were arbitrated by a third party. The researchers had obtained GCP certification and been trained in accordance with unified standards and methods.
Assessment of alternative descriptions
According to the scoring system for alternative descriptions [27], an ICF scored 0 points if it did not mention any alternatives, 1 point if it mentioned alternatives but contained no description, 2 points if it mentioned and described alternatives, and 3 points if it listed or described alternatives in detail. Similarly, two researchers independently scored all documents and resolved disagreements.
Data analysis
Statistical analyses were performed with SPSS 19.0 software (SPSS Inc., Chicago, IL, USA). Continuous variables were presented as the median with the range. Skewed data were submitted to the Mann-Whitney U test, and count data were submitted to the chi-square test. Note that P values < 0.05 were considered statistically significant.
Results
Characteristics of ICFs in China
A total of 334 ICFs from the ethics committee database were obtained. Among these forms, 179 ICFs were excluded from our study for the following reasons: 87 were bioequivalence studies, 35 were pharmacokinetic studies, 24 were medical device clinical trials, 20 were phase I clinical studies, 1 was tolerance research, 4 involved questionnaires, 3 involved medical techniques, and 5 were scanned documents. Finally, 155 ICFs were included in our study (104 Chinese local ICFs and 51 international ICFs), and the characteristics of all ICFs are shown in Table 1.
Readability and content assessment of ICFs used in China
The median number of pages for the ICFs used in China was 7 pages (3–25), with 5286 (1808–21302) words (Table 2). A total of 97.4% of ICFs used appropriate section headings, and only 9.0% of ICFs inserted flow charts explaining how the study would be conducted. The most commonly used font sizes were 12-point font and 10.5-point font. The median readability score was 4.31 (4.02–4.41), with 63.9% at the 2nd level and 36.1% at the 3rd level (Table 3). Of the 27 required elements, 5 elements were frequently neglected (occurrence rate ≤50%), including an explanation of informed consent, follow-up processing of the data/sample, inclusion/exclusion criteria, double blinding, and unpredictable risks (Table 4). None of the ICFs contained all analyzed elements. Specifically, 7.7% of the ICFs included more than 25 elements, 47.1% included 21–24 elements, 37.4% included 16–20 elements, and 7.8% included 15 or fewer elements. The average score for the description of alternatives to participation was 1.06, and 27.7% of ICFs did not mention any alternatives (Table 5).
Comparison between Chinese local ICFs and international ICFs
Comparisons of the readability, content, and description of alternatives to participation between Chinese local ICFs and international ICFs are shown in Tables 2–5. Chinese local ICFs were significantly shorter than international ICFs (P < 0.001). Most Chinese local and international ICFs used subtitles to clarify details (97.1% vs. 98.0%, respectively, P > 0.05). International ICFs were more likely to use illustrations than Chinese local ICFs (23.5% vs. 1.9%, respectively, P < 0.001). A significant font size difference between the Chinese local ICFs and international ICFs was found. In addition, the readability score for international ICFs was higher than that for Chinese local ICFs (P < 0.001). A total of 86.3% of international ICFs and 52.9% of Chinese local ICFs were at “2nd easy” level (Table 3). Compared with Chinese local ICFs (most contained 18–21 elements), international ICFs (most contained 22–25 elements) had more of the required items. The average score for the description of alternatives was 1.51 for international ICFs, which was higher than that for Chinese local ICFs (Table 5).
Discussion
To the best of our knowledge, the current study was the first to use a readability formula and a content evaluation tool for the readability and content analysis of ICFs used in China. This initial attempt allowed us to explore the defects of these ICFs in detail and to explore the differences between Chinese local and international ICFs. The content evaluation tool included comprehensive information and considered the extent to which the forms contained key information that was useful for subjects to make decisions. Each ICF was rated by two researchers independently, thereby avoiding rater bias.
The results showed that the median readability score of the evaluated ICFs was 4.31 at a 3rd level, with 63.9% at the 2nd level and 36.1% at the 3rd level. The education level of Chinese resident was relatively low, with 37.46% at or below the primary school level and 36.18% at the secondary school level [23], leading to the reading difficulties for the participants. Similarly, other studies found that ICFs from the United States, New Zealand, France, Spain, and Croatia failed to offer subjects complete information using clear and direct language [11, 16–18, 28, 29]. 97% of Canadian ICFs used appropriate section headings to label the separate sections of content, and such headings were helpful for reading comprehension, and the results are just consistent with ours [16]. Additionally, multinational pharmaceutical companies usually included flow charts visually depicting how the study would be conducted and markedly drawing attention to the materials to improve comprehension among participants with low literacy levels [30, 31]. Our results indicated that 12-point font and 10.5-point font were the most common font sizes of Chinese ICFs, whereas 11-point font was typically used in English ICFs [16], indicating that larger fonts improve the understanding of ICFs especially for the elderly subjects and those with poor eyesight.
Compared with Chinese local ICFs, international ICFs had better readability, contained more detailed and complete content. Generally, people are not willing to spend much time reading consent forms, and they are likely to read no more than 4 pages of an ICF [32, 33], but the median number of pages for the international ICFs was 14, which was beyond the accepted range of normal. Strategies for reducing the length of ICFs while concisely retaining all essential elements are worth exploring in both Chinese local ICFs and international ICFs. Among the ICFs reviewed, we found that the majority did not perform well according to the content evaluation tool. Our results found that ICFs used in China did not provide full information disclosure, and 7.8% of the ICFs contained 15 or fewer elements. Detailed explanations of these required items could help subjects to weigh the advantages and disadvantages and better safeguard their interests [34]. Incredibly, ICFs used in China attached great importance to the contact information of the researcher but not to the contact information of the ethics committee. Chinese local studies have promised direct benefits to participants but have neglected the description of social benefits, which prevents participants from realizing the social value of clinical research, such as the promotion of medical developments. Only 45.8% of the ICFs used in China emphasized the unpredictable risks of clinical trials. Researchers may intend to minimize patient concerns by omitting these serious risks, but this is considered unacceptable [16]. No unified template or sample text is provided by law, in ordinances or in guidelines to clearly define all the necessary elements of ICFs may lead to the poor quality of Chinese ICFs.
Based on the results from our study, ethics committees and other reviewing institutions should establish standards for the length, readability and content of consent documents. ICFs should theoretically be limited to no more than 4200 characters (approximately 6 pages). According to the findings by Liao [35], a subject can read Chinese at a rate of 562 to 622 words per minute, and the average person would likely read the document in 5 to 7 minutes [15]; thus, limiting the length of ICFs to 4200 characters is reasonable. However, in a real-world situation, restricting ICFs to 6 pages may assume the risk of presenting incomplete information to participants in complex trials; thus, fewer words should be used to state more information in order to be concise and succinct in ICFs. Ethics committees should develop Chinese readability software. Every consent document should be checked by the same software prior to submission to ensure that ICFs are at or below the 2nd level. We encourage the use of section headings and illustrations, which improve comprehension for participants. To ensure content integrity, ethics committees should offer templates and sample text to investigators, and these documents must be written in clear and direct language to ensure that subjects understand them accurately. Moreover, we should improve the process of ethical review by implementing consistent training practices for consenters.
Several limitations of this study must be considered. First, our sample was relatively small and all the ICFs were obtained from one single ethics committee. Although all ICFs were sampled randomly to ensure representativeness, it was difficult to ensure complete consistency among the ICFs used in China. Further study should be carried out to assess ICFs from multiple ethics committees. Second, compared with the FRG and GFI, which have been used to assess English readability for many years, the Chinese readability formula has not been generalized and applied widely, even though it was invented in 1974. We were unable to use a computer program with readability formulas adapted for the Chinese language, and the manual review method increased the likelihood of random error. Third, we did not assessed readability separately for each section, including general items, rights of subjects, scientific aspects, and ethical aspects. Only the first draft of ICFs was included in the analysis which may have caused selective bias. Fourth, the content measurement may not accurately reflect the perspectives of patients who read these ICFs.
Conclusion
The ICFs used in China were difficult to read for most participants. And these documents had poor description of alternatives to participation, and failed to provide a high degree of information disclosure, including an explanation of informed consent, follow-up processing of the data/sample, inclusion/exclusion criteria, double blinding, and unpredictable risks. International ICFs had better readability and content integrity than Chinese local ICFs. Greater efforts should be devoted to improving the readability and content of written documents to ensure the rights of potential participants.
Acknowledgments
The authors would like to thank the Third Xiangya Hospital Ethics Committee for providing all the information of informed consent forms.
Author Contributions
- Conceptualization: HY.
- Data curation: XW.
- Funding acquisition: GW YL HY.
- Investigation: JS NX RC.
- Methodology: YL GW.
- Project administration: HY.
- Resources: YX XW.
- Validation: ZH GY.
- Writing – original draft: GW.
- Writing – review & editing: GW XL LH YL HY.
References
- 1. Harmer A, Xiao Y, Missoni E, Tediosi F. 'BRICS without straw'? A systematic literature review of newly emerging economies' influence in global health. Globalization Health. 2013;9: 15. pmid:23587342
- 2. Chakma J, Sun GH, Steinberg JD, Sammut SM, Jagsi R. Asia's ascent—global trends in biomedical R&D expenditures. N Engl J Med. 2014;370: 3–6. pmid:24382062
- 3. Gillies K, Huang W, Skea Z, Brehaut J, Cotton S. Patient information leaflets (PILs) for UK randomised controlled trials: a feasibility study exploring whether they contain information to support decision making about trial participation. Trials. 2014;15: 62. pmid:24548781
- 4. Foe G, Larson EL. Reading Level and Comprehension of Research Consent Forms: An Integrative Review. J Empir Res Hum Res Ethics. 2016;11: 31–46. pmid:27106889
- 5. Flory J, Emanuel E. Interventions to improve research participants' understanding in informed consent for research: a systematic review. JAMA. 2004;292: 1593–1601. pmid:15467062
- 6. Joffe S, Cook EF, Cleary PD, Clark JW, Weeks JC. Quality of informed consent in cancer clinical trials: a cross-sectional survey. Lancet. 2001;358: 1772–1777. pmid:11734235
- 7. Daugherty C, Ratain MJ, Grochowski E, Stocking C, Kodish E, Mick R, et al. Perceptions of cancer patients and their physicians involved in phase I trials. J Clin Oncol. 1995;13: 1062–1072. pmid:7738612
- 8. Meropol NJ, Weinfurt KP, Burnett CB, Balshem A, Benson AB, Castel L, et al. Perceptions of patients and physicians regarding phase I cancer clinical trials: implications for physician-patient communication. J Clin Oncol. 2003;21: 2589–2596. pmid:12829680
- 9. Griffin JM, Struve JK, Collins D, Liu A, Nelson DB, Bloomfield HE. Long term clinical trials: how much information do participants retain from the informed consent process? Contemp Clin Trials. 2006;27: 441–448. pmid:16798101
- 10. Friedman DB, Kao EK. A comprehensive assessment of the difficulty level and cultural sensitivity of online cancer prevention resources for older minority men. Prev Chronic Dis. 2008;5: A07. pmid:18081996
- 11. Vučemilo L, Borovečki A. Readability and content assessment of informed consent forms for medical procedures in Croatia. PLOS ONE. 2015;10: e0138017. pmid:26376183
- 12. Stryker JE, Wray RJ, Emmons KM, Winer E, Demetri G. Understanding the decisions of cancer clinical trial participants to enter research studies: factors associated with informed consent, patient satisfaction, and decisional regret. Patient Educ Couns. 2006;63: 104–109. pmid:16242898
- 13. Paasche-Orlow MK, Taylor HA, Brancati FL. Readability standards for informed-consent forms as compared with actual readability. N Engl J Med. 2003;348: 721–726. pmid:12594317
- 14. Grossman SA, Piantadosi S, Covahey C. Are informed consent forms that describe clinical oncology research protocols readable by most patients and their families? J Clin Oncol. 1994;12: 2211–2215. pmid:7931491
- 15. Sharp SM. Consent documents for oncology trials: does anybody read these things? Am J Clin Oncol. 2004;27: 570–575. pmid:15577434
- 16. Cheung WY, Pond GR, Heslegrave RJ, Enright K, Potanina L, Siu LL. The contents and readability of informed consent forms for oncology clinical trials. Am J Clin Oncol. 2010;33: 387–392. pmid:19884804
- 17. Ménoni V, Lucas N, Leforestier JF, Doz F, Chatellier G, Jacqz-Aigain E, et al. Readability of the written study information in pediatric research in France. PLOS ONE. 2011;6: e18484. pmid:21494689
- 18. Benatar JR, Mortimer J, Stretton M, Stewart RA. A booklet on participants' rights to improve consent for clinical research: a randomized trial. PLOS ONE. 2012;7: e47023. pmid:23094034
- 19. Flesch R. A new readability yardstick. J Appl Psychol. 1948;32: 221–233. pmid:18867058
- 20.
Gunning R. The technique of clear writing. McGraw-Hill; 1968.
- 21. Yang XJ. The readabilty formula for Chinese newspaper. Mass Commun Res. 1974;13: 55.
- 22. Fry E. Readability versus leveling: both of these procedures Can Help teachers select books for readers at different stages. Reading Teach. 2002;56: 286–292.
- 23. National Health and Family Planning Commission(NHFPC). The level of health literacy of Chinese residents in 2012:surveillance results. Chin J Health Educ. 2015;31: 99–103.
- 24. Koonrungsesomboon N, Teekachunhatean S, Hanprasertpong N, Laothavorn J, Na-Bangchang K, Karbwang J. Improved participants’ understanding in a healthy volunteer study using the SIDCER informed consent form: a randomized-controlled study. Eur J Clin Pharmacol. 2016;72: 413–42121. pmid:26713336
- 25. Tam NT, Huy NT, Thoa le TB, Long NP, Trang NT, Hirayama K, et al. Participants' understanding of informed consent in clinical trials over three decades: systematic review and meta-analysis. Bull World Health Organ. 2015;93: 186–98H. pmid:25883410
- 26. Pandiya A. Readability and comprehensibility of informed consent forms for clinical trials. Perspect Clin Res. 2010;1: 98–100. pmid:21814628
- 27. Resnik DB, Patrone D, Peddada S. Evaluating the quality of information about alternatives to research participation in oncology consent forms. Contemp Clin Trials. 2010;31: 18–21. pmid:19897054
- 28. Calle-Urra JE, Parra-Hidalgo P, Saturno-Hernández P, Fonseca-Miranda Y, Martínez-Martínez MJ. Evaluation and improvement of the informed consent documents in 9 hospitals, Spain. Rev Esp Salud Publica. 2015;89: 307–319. pmid:26388344
- 29. Ruiz Ramos J, Cueto-Sola M, García Robles A, Pérez Huertas P, Tordera Baviera M, Poveda Andrés JL. Quality and legibility of written informed consent form in pharmacogenetic research. Cuad Bioet. 2015;26: 129–138. pmid:26030019
- 30. Houts PS, Witmer JT, Egeth HE, Loscalzo MJ, Zabora JR. Using pictographs to enhance recall of spoken medical instructions II. Patient Educ Couns. 2001;43: 231–242. pmid:11384821
- 31. Houts PS, Doak CC, Doak LG, Loscalzo MJ. The role of pictures in improving health communication: a review of research on attention, comprehension, recall, and adherence. Patient Educ Couns. 2006;61: 173–190. pmid:16122896
- 32. Sharp SM. Improving the process of informed consent. Appl. Clin Res. 2001;10: 74–82.
- 33. Brockett RG. Developing written learning materials: A proactive approach. Lifelong Learn. 1983;7: 16.
- 34. Cahana A, Hurst SA. Voluntary informed consent in research and clinical care: an update. Pain Pract. 2008;8: 446–451. pmid:19000172
- 35. Liao JQ, Zhang WS. The study of Chinese reading speed. Chin J Ergon. 1996;1: 38–41.