Quantitative tools have been developed to evaluate the readability of written documents and have been used in several studies to evaluate information and consent forms. These studies all showed that such documents had a low level of readability. Our objective is to evaluate the readability of Information and Consent Forms (ICFs) used in clinical research.
Methods and Findings
Clinical research protocols were collected from four public clinical research centers in France. Readability was evaluated based on three criteria: the presence of an illustration, the length of the text and its Flesch score. Potential effects of protocol characteristics on the length and readability of the ICFs were determined. Medical and statutory parts of the ICF form were analyzed separately. The readability of these documents was compared with that of everyday contracts, press articles, literary extracts and political speeches. We included 209 protocols and the corresponding 275 ICFs. The median length was 1304 words. Their Flesch readability scores were low (median: 24), and only about half that of selected press articles. ICF s for industrially sponsored and randomized protocols were the longest and had the highest readability scores. More than half (52%) of the text in ICFs concerned medical information, and this information was statistically (p<0.05) more readable (Flesch: 28) than statutory information (Flesch: 21).
Citation: Ménoni V, Lucas N, Leforestier JF, Dimet J, Doz F, Chatellier G, et al. (2010) The Readability of Information and Consent Forms in Clinical Research in France. PLoS ONE 5(5): e10576. https://doi.org/10.1371/journal.pone.0010576
Editor: Margaret Sampson, Children's Hospital of Eastern Ontario, Canada
Received: October 8, 2009; Accepted: March 31, 2010; Published: May 11, 2010
Copyright: © 2010 Ménoni et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This study was funded by the Assistance Publique des Hopitaux de Paris as a Programme Hospitalier de Recherche Clinique. The funder played no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
The first step in taking part in a clinical trial is, for the patient or the healthy volunteer, to express informed consent    . This requires receiving and understanding potentially complex scientific information. By law, obtaining consent for research must include written information and agreement following a discussion between the physician and their patient. The written part of information, delivered as an Information and Consent Form (ICF), must be both exhaustive and understandable. Quantitative tools have been developed to evaluate the readability of written documents , and have been used in several studies to evaluate ICFs           . These studies all showed that such documents had a low level of readability. However, these publications were mostly aimed at specific fields of research (oncology, imaging analysis, surgery, pediatrics). We evaluated the readability of a broad range of ICFs used in clinical research protocols from various fields of research, involving diverse populations and all phases of clinical development.
Collection of research protocols and their ICFs
All interventional protocols involving the four public clinical research centers that received authorization by the Comité de Protection des Personnes (Institutional Review Board, or IRB) were included between 2001 and 2008. Protocols which did not require authorization from an IRB (data based) were excluded. The protocols conducted in these four centers include those with industrial and institutional sponsors, and most are multicenter studies generally involving other French clinical research structures. Protocols were categorized into three groups depending on their goal (therapeutic, pathophysiological or epidemiological). For each protocol, we determined: the type of sponsor (industrial or institutional), the year that the IRB granted authorization (to determine whether the protocol was submitted under the new French legislation  on research, resulting from the application of the European directive 2001/20/EC ), the field of medical research, the phase of the study, the inclusion of pediatric patients, whether or not a randomization procedure was used, whether or not the protocol was potentially life-threatening, and whether or not the protocol involved invasive tests (i.e. procedures, other than taking blood, which require bodily injury or which are painful). Documents not statutorily necessary and provided for those not legally responsible (minors or individuals incapable of making a decision) were also excluded. In cases where an information sheet and a consent form were given as two separate documents, the two documents were grouped together for overall analysis of the written information given to the patient. Each ICF was saved and three different analyses were conducted: one of the entire original text, one of only the paragraphs dealing with medical information and one of only the paragraphs dealing with statutory information.
Three criteria were used to evaluate readability: Flesch readability score, length of the text and the presence of illustrations. The Flesch score  was calculated using the equation: 206.835− (1.015sl) − (0.846wl), where sl is the average sentence length (mean number of words per sentence) and wl is the average word length (mean number of syllables per word). The resulting score lies between 0, for texts that are not easily understood, and 100, for an easily understandable text. This score can be calculated with Microsoft Word® software for texts written in English, but this software is not suitable for use with French, probably due to problems with counting syllables. We therefore developed a Flesch index calculator for the French language using a PERL script, which is now freely available (http://search.cpan.org/~leforesjf/Lingua-FR-Fathom-0.01/).
We also analyzed, as internal controls, a set of texts classified into four categories:
- Standard, everyday contracts (rental agreements, marriage documents, work contracts and general sales contracts for electricity, telephone and railway networks).
- Press articles taken from the two most widely distributed free newspapers (20 minutes and Metro), from the two most widely sold newspapers in France (Le Monde and l'Equipe) and from a popular medical health magazine (Santé magazine).
- Literary extracts from French authors of the 19th century (Colette, Dumas, Hugo, Maupassant, Musset, Proust, Rimbaud, Stendhal and Zola).
- Politicians' speeches (by the five candidates who received most votes in the 2007 French presidential election).
Details of the protocols were entered into a Microsoft Access® database. The ICF computer files were collected and analyzed using the PERL script that we had developed (figure 1). The results obtained were entered into the database described above containing the characteristics of the corresponding protocols. Statistical analyses were performed using the NCSS® software. Results are expressed as median and 25th and 75th percentiles. For the number of words, none of the variables analyzed has a parametric distribution, so medians were compared using a Mann-Whitney test for pair-wise comparisons or a Kruskal Wallis test when more than two variables were being tested. For the Flesch scores, the variables analyzed could be compared with a t-test. Prism Graph Pad® software was used to produce graphs for comparisons between ICFs and control texts.
It includes three steps: the first is applicable to all European languages. The second step is specific to the language considered. The last step is the calculation of the Flesch index with the variables obtained from the previous steps.
We included 209 protocols from 40 specialist medical fields, mainly oncology (33%), hematology (18%) and paediatrics (12%). Some protocols had several ICFs addressing different populations (adult patients, parents, guardians or proxy, such that we collected and analyzed all 275 ICFs used in these protocols. The goal of the protocols included was mostly therapeutic (65%), most were institutionally sponsored (82%) and the majority were national (74%). Sixty-six (32%) of the protocols planned to include patients under the age of 18 years.
Twenty of the 275 ICFs analyzed (7%) included a diagram or table explaining the method used in the trial or the schedule of visits. None of the everyday contracts, politicians' speeches (given orally) or the literacy extracts (only historical texts were included) contained illustrations. By contrast, all of the press articles analyzed were illustrated.
The median length of ICFs analyzed was 1304 words (991;1928), equivalent to five pages. The variables studied that had a significant influence on the length of the ICF are shown in table 1. No other factors analyzed (year of authorization by an institutional review board, the phase of the study, the inclusion of paediatric patients, or field of medical research) had an effect on ICF length. Overall, ICFs used in industrially sponsored protocols were more than twice as long as those used in institutionally sponsored protocols. Similarly, ICFs used in therapeutic protocols were one-and-a-half times longer than those used in epidemiological or psychopathological protocols. ICFs used in randomized protocols were longer than those used in open protocols. ICFS in protocols involving a risk for the patient (identified in our study as potentially life-threatening or involving invasive tests) were generally longer than those in protocols involving only minimal risks.
The median Flesch score for the entire set of ICFs was 24 (21;28). There was little variability of this score within the set of ICFs. The influence of selected factors on readability scores is shown in table 1. Factors associated with good readability were: epidemiological protocols, industrial sponsorship and use of randomization procedures. However, despite being statistically significant, the effects of these variables on the Flesch score were only weak.
Of the total content of ICFs, 52% was medical. The readability score of the medical sections was 28 (25;30), which was significantly higher (p<0.05) than the value of 21 (16;27) for the statutory sections.
Comparison with the selected reference texts (figure 2) showed that the readability scores obtained for the ICFs were similar to those obtained for everyday contracts: 30 (24; 33), and significantly lower than scores obtained for literary extracts: 62 (48; 70), political speeches: 47 (41; 50) and press articles: 41(32; 50).
Central bars correspond to the medians, the ends of the boxes to the 25th and 75th percentiles and ends of the bars to the maximum and minimum values. ICF: Information and Consent Form. N.S: Not Significant. ICFs (N = 275) have significantly (p<0.05) lower readability scores than literary extracts (N = 9), political speeches (N = 5) and articles from the press (N = 51), but similar scores to everyday contracts (N = 6).
Our analysis of 209 protocols and the 275 corresponding ICFs showed a poor level of readability of these documents, as assessed using three criteria: the length of the text, the Flesch score and the presence of an illustration.
Only a small proportion of the ICFs studied included an illustration, a diagram or a table. A previous study carried out in an emergency service  found that patients showed a better understanding when the information documents contained illustrations. Similarly, Tait et al.  showed that patients in research trials preferred illustrated ICFs.
The Flesch scores calculated for the reference texts were highly variable, showing the sensitivity of this test: the scores varied greatly depending on the type of document analyzed. In contrast, the scores for the ICFs analyzed were mostly similar, despite the diversity of protocols included (40 different fields of medical research were represented, with both adult and pediatric protocols, and both institutional and industrial sponsorship).
The readability of the ICFs was similar to that of everyday contracts, which are renowned for their particular lack of clarity.
The Flesch scores obtained in this analysis were similar to those reported in the only other French study addressing this topic , in which 73 ICFs were analyzed. French ICFs were less readable than ICF written in English, where scores of between 30 and 55 were obtained         . Among the factors studied, we found industrial sponsorship to be associated with ICFs being both twice as long and more readable. This effect was not observed by Paris et al. . However, they only included a small number of industrially sponsored protocols in their analysis (N = 18).
The use of randomization procedures was associated in our study with longer and more readable ICFs. The increased length of text, as previsouly reported by Paris et al. , may be because the concept of randomization needs to be described to patients, the majority of whom know very little about this procedure. The better readability obtained for ICFs for randomized protocols may thus be indicative of the efforts of the investigators to explain this concept on the ICF.
As previously shown by Mader et al.  for emergency medicine research protocols, we report that the length of the ICFs increased with protocol risk. Furthermore, we also found that the implementation of new biomedical research legislation, in 2006 , was associated with increased length of ICFs, but not with improved readability.
We demonstrate that the statutory sections in the ICFs were significantly less readable than the medical sections. This is despite medical information tending to contain very long words, reducing the Flesch score. This finding is consistent with a previous study  in which Paasche-Orlow et al. found a low level of readability for examples of statutory information made available to investigators by several institutional review boards.
There were two biases in our study. First, we included only those ICFs available in computer files; this allowed us to include eighty percent of all the studies conducted in the various collaborating centers. The second bias is intrinsic to the readability score used. The Flesch score evaluates a text on the basis of the length of sentences and words alone, but nevertheless allows a rapid, objective and quantitative evaluation of the complexity of a document. The use of the Flesch score to assess the readability of a text has been validated by several studies    . This score itself does not reflect the level of patient understanding, because the understanding of any particular individual depends on intrinsic factors (for example, first language, culture and level of education). Nevertheless, two previous studies   have demonstrated that patients' (adults and children) understanding of a protocol is improved when given an ICF that has a better readability. It has also been shown   that the addition of illustrations to ICFs helps improve both understanding and acceptance of protocols by participants. However, there are no norms concerning optimal Flesch scores, numbers of words or of illustrations for ICFs; a compromise must be found between the readability of the ICF and the quantity and complexity of the information to be delivered. Identifying the best use of tools for achieving the competing goals of comprehensive and understandable information in consent forms was not among the objectives of our study. A second, interventional study is underway with this aim.
Our study showed that the readability of ICFs used in clinical research is poor, regardless of the type of protocol. The systematic use of the Flesch score may be envisaged as a way of improving readability and thus the understanding of ICFs. Certain institutional review boards have already implemented this idea . However, a prospective analysis of this test in French should be carried out before it is put into general use.
Conceived and designed the experiments: VM JMT HC. Performed the experiments: VM JFL JD FD GC JMT HC. Analyzed the data: VM NL JFL GC JMT HC. Contributed reagents/materials/analysis tools: VM NL JD FD GC JMT HC. Wrote the paper: VM JMT HC.
- 1. International Conference on Harmonisation of technical requirements for registration of pharmaceuticals for human use (1996) International Conference on Harmonisation of technical requirements for registration of pharmaceuticals for human use1996Guideline for Good Clinical Practice. E6 R1. June 1996. Available: http://www.ich.org/. Guideline for Good Clinical Practice. E6 R1. June 1996. Available: http://www.ich.org/.
- 2. World Medical Association (2008) World Medical Association2008“Declaration of Helsinki: ethical principles for research involving human subjects”, October 2008. Available: http://www.wma.net/. “Declaration of Helsinki: ethical principles for research involving human subjects”, October 2008. Available: http://www.wma.net/.
- 3. World Health Organization and CIOMS (1981) World Health Organization and CIOMS1981The WHO/CIOMS guidelines, proposed international guidelines for biomedical research involving human subjects, 1981. Availabe: http://www.tbethics.org/. The WHO/CIOMS guidelines, proposed international guidelines for biomedical research involving human subjects, 1981. Availabe: http://www.tbethics.org/.
- 4. Additional protocol to the convention on human rights and biomedicine, concerning biomedical research, January 2005. Available: http://conventions.coe.int/. Additional protocol to the convention on human rights and biomedicine, concerning biomedical research, January 2005. Available: http://conventions.coe.int/.
- 5. Bailin A, Grafstein A (2001) The linguistic assumptions underlying readability formulae: a critique. Language and communication 21: 285–301.A. BailinA. Grafstein2001The linguistic assumptions underlying readability formulae: a critique.Language and communication21285301
- 6. Grundner TM (1980) On the readability of surgical consent forms. N Engl J Med 302: 900–902.TM Grundner1980On the readability of surgical consent forms.N Engl J Med302900902
- 7. Morrow GR (1980) How readable are subject consent forms? Jama 244: 56–58.GR Morrow1980How readable are subject consent forms?Jama2445658
- 8. Tarnowski KJ, Allen DM, Mayhall C, Kelly PA (1990) Readability of pediatric biomedical research informed consent forms. Pediatrics 85: 58–62.KJ TarnowskiDM AllenC. MayhallPA Kelly1990Readability of pediatric biomedical research informed consent forms.Pediatrics855862
- 9. Hopper KD, TenHave TR, Hartzel J (1995) Informed consent forms for clinical and research imaging procedures: how much do patients understand? AJR Am J Roentgenol 164: 493–496.KD HopperTR TenHaveJ. Hartzel1995Informed consent forms for clinical and research imaging procedures: how much do patients understand?AJR Am J Roentgenol164493496
- 10. Grossman SA, Piantadosi S, Covahey C (1994) Are informed consent forms that describe clinical oncology research protocols readable by most patients and their families? J Clin Oncol 12: 2211–2215.SA GrossmanS. PiantadosiC. Covahey1994Are informed consent forms that describe clinical oncology research protocols readable by most patients and their families?J Clin Oncol1222112215
- 11. Philipson SJ, Doyle MA, Gabram SG, Nightingale C, Philipson EH (1995) Informed consent for research: a study to evaluate readability and processability to effect change. J Investig Med 43: 459–467.SJ PhilipsonMA DoyleSG GabramC. NightingaleEH Philipson1995Informed consent for research: a study to evaluate readability and processability to effect change.J Investig Med43459467
- 12. White LJ, Jones JS, Felton CW, Pool LC (1996) Informed consent for medical research: common discrepancies and readability. Acad Emerg Med 3: 745–750.LJ WhiteJS JonesCW FeltonLC Pool1996Informed consent for medical research: common discrepancies and readability.Acad Emerg Med3745750
- 13. Mader TJ, Playe SJ (1997) Emergency medicine research consent form readability assessment. Ann Emerg Med 29: 534–539.TJ MaderSJ Playe1997Emergency medicine research consent form readability assessment.Ann Emerg Med29534539
- 14. Sharp SM (2004) Consent documents for oncology trials: does anybody read these things? Am J Clin Oncol 27: 570–575.SM Sharp2004Consent documents for oncology trials: does anybody read these things?Am J Clin Oncol27570575
- 15. Paris A, Cracowski JL, Ravanel N, Cornu C, Gueyffier F, et al. (2005) Readability of informed consent forms for subjects participating in biomedical research: updating is required. Presse Med 34: 13–18.A. ParisJL CracowskiN. RavanelC. CornuF. Gueyffier2005Readability of informed consent forms for subjects participating in biomedical research: updating is required.Presse Med341318
- 16. Christopher PP, Foti ME, Roy-Bujnowski K, Appelbaum PS (2007) Consent form readability and educational levels of potential participants in mental health research. Psychiatr Serv 58: 227–232.PP ChristopherME FotiK. Roy-BujnowskiPS Appelbaum2007Consent form readability and educational levels of potential participants in mental health research.Psychiatr Serv58227232
- 17. Loi n°2004-806 du 9 août 2004 sur la recherche biomédicale. Code de la Santé Publique livre 1 titre 2. Available: http://www.legifrance.fr. Loi n°2004-806 du 9 août 2004 sur la recherche biomédicale. Code de la Santé Publique livre 1 titre 2. Available: http://www.legifrance.fr.
- 18. Directive 2001/20/EC of the European parliament and of the council of 4 april 2001 on the approximation of the laws, regulations and administrative provisions of the member states relating to the implementation of good clinical practice in the conduct of clinical trials on medicinal products for human use. Available: http://europa.eu.int. Directive 2001/20/EC of the European parliament and of the council of 4 april 2001 on the approximation of the laws, regulations and administrative provisions of the member states relating to the implementation of good clinical practice in the conduct of clinical trials on medicinal products for human use. Available: http://europa.eu.int.
- 19. Flesch R (1948) A new readability yardstick. J Appl Psychol 32: 221–233.R. Flesch1948A new readability yardstick.J Appl Psychol32221233
- 20. Austin PE, Matlack R 2nd, Dunn KA, Kesler C, Brown CK (1995) Discharge instructions: do illustrations help our patients understand them? Ann Emerg Med 25: 317–320.PE AustinR. Matlack 2ndKA DunnC. KeslerCK Brown1995Discharge instructions: do illustrations help our patients understand them?Ann Emerg Med25317320
- 21. Tait AR, Voepel-Lewis T, Malviya S, Philipson SJ (2005) Improving the readability and processability of a pediatric informed consent document: effects on parents' understanding. Arch Pediatr Adolesc Med 159: 347–352.AR TaitT. Voepel-LewisS. MalviyaSJ Philipson2005Improving the readability and processability of a pediatric informed consent document: effects on parents' understanding.Arch Pediatr Adolesc Med159347352
- 22. Paasche-Orlow MK, Taylor HA, Brancati FL (2003) Readability standards for informed-consent forms as compared with actual readability. N Engl J Med 348: 721–726.MK Paasche-OrlowHA TaylorFL Brancati2003Readability standards for informed-consent forms as compared with actual readability.N Engl J Med348721726
- 23. Paris A, Nogueira da Gama Chaves D, Cornu C, Maison P, Salvat-Mélis M, et al. (2007) Improvement of the comprehension of written information given to healthy volunteers in biomedical research: a single-blind randomized controlled study. Fundam Clin Pharmacol 21: 207–214.A. ParisD. Nogueira da Gama ChavesC. CornuP. MaisonM. Salvat-Mélis2007Improvement of the comprehension of written information given to healthy volunteers in biomedical research: a single-blind randomized controlled study.Fundam Clin Pharmacol21207214