Conceived and designed the experiments: TS NEK. Performed the experiments: TS BC UM. Analyzed the data: LHC. Wrote the paper: LHC NEK TS.
The authors have declared that no competing interests exist.
Informed consent (IC) has been an international standard for decades for the ethical conduct of clinical trials. Yet frequently study participants have incomplete understanding of key issues, a problem exacerbated by language barriers or lack of familiarity with research concepts. Few investigators measure participant comprehension of IC, while even fewer conduct interim assessments once a trial is underway.
We assessed comprehension of IC using a 20-question true/false quiz administered in 6-month intervals in the context of a placebo-controlled, randomized trial for the prevention of tuberculosis among HIV-infected adults in Botswana (2004–2009). Quizzes were offered in both Setswana and English. To enroll in the TB trial, participants were required to have ≥16/20 correct responses. We examined concepts understood and the degree to which understanding changed over three-years. We analyzed 5,555 quizzes from 1,835 participants. The participants' highest education levels were: 28% primary, 59% secondary, 9% tertiary and 7% no formal education. Eighty percent of participants passed the enrollment quiz (Quiz1) on their first attempt and the remainder passed on their second attempt. Those having higher than primary education and those who took the quiz in English were more likely to receive a passing score on their first attempt (adjusted odds ratios and 95% confidence intervals, 3.1 (2.4–4.0) and 1.5 (1.2, 1.9), respectively). The trial's purpose or procedures were understood by 90–100% of participants, while 44–77% understood randomization, placebos, or risks. Participants who failed Quiz1 on their initial attempt were more likely to fail quizzes later in the trial. Pass rates improved with quiz re-administration in subsequent years.
Administration of a comprehension quiz at enrollment and during follow-up was feasible in a large, international collaboration and efficiently determined IC comprehension by trial participants. Strategies to improve understanding of concepts like placebos and randomization are needed. Comprehension assessments throughout a study may reinforce key concepts.
The protection of human subjects in medical research is a fundamental concern of bioethics. Informed consent is one means of providing such protection. Through the delivery of key study information, informed consent allows potential participants, based on their own values, to decide which risks, benefits, and procedures are acceptable to them. While informed consent is expected to be a dialogue between researcher and potential participant, consent information is typically provided through a written consent form as well, which is also used to document the participant's agreement to enroll in the study.
While informed consent documents are useful as a means of standardizing information, research participants in multiple settings and in multiple studies have been shown to have incomplete or inaccurate understanding of many facets of information provided through the informed consent process
Further hampering research understanding is the decrease in the ability to remember study information over time
A clinical trial conducted in Botswana from 2004–2009 provided an opportunity to examine how well participants understood study information when first told about the study, and to measure their understanding over time. The Isoniazid Prevention Therapy (IPT) trial, part of an ongoing collaboration between the U.S. Centers for Disease Control and Prevention (CDC) and the Botswana Ministry of Health, was a randomized, double-blind, placebo-controlled study of limited (6 months) vs. continuous (36 months) IPT among HIV-infected persons ≥18 years of age. This three-year study allowed investigators to detect any differences in rates of active TB incidence between the two study arms. Participation involved monthly clinic visits over the 36 months. The informed consent form for the trial was first developed in English by study investigators, translated into Setswana, and back-translated into English to ensure consistency of meaning.
In an effort to ensure that participants adequately understood study information, investigators developed a 20 item true/false comprehension quiz to administer following the informed consent discussion, but prior to enrollment. To be able to join the study, participants had to be able to answer at least 16 of the 20 items correctly on the quiz. Potential participants who failed further discussed the protocol with study nurses, and were then allowed to repeat the quiz. Those who failed the quiz twice were not allowed to enroll in the clinical trial.
This manuscript reports on the analysis of the comprehension quiz data. Specifically, we examined the extent to which participants obtained a passing quiz score at enrollment, factors associated with passing or failing scores, likelihood of retaining study information over time, and which questions were most likely to be answered correctly and incorrectly.
Recruitment for the IPT trial took place from 2004 through 2006. In December 2006, the investigators amended the study protocol such that the comprehension test was re-administered to study subjects every 6 months throughout the remainder of their time in the trial, allowing investigators to measure participants' understanding of the study over time. Participants were not removed from the study at this stage if a passing score (≥16/20 correct responses) was not obtained, but nurses reviewed relevant information with participants, allowing reinforcement of information that had been either forgotten or misunderstood. As the decision to re-administer the quiz every 6 months was made 2 years after enrollment began, participants who enrolled in 2004 received their first follow-up quiz about 2 years into the study (and subsequently at months 30 and 36). Subjects who first enrolled in 2006 were re-quizzed every 6 months after enrollment, for a total of up to 6 quizzes.
Data for this study consisted of all available enrollment and follow-up quizzes from participants in the IPT trial. Data were abstracted from the hard copy paper quizzes in July 2008 from the 2 study sites in Gaborone and Francistown, Botswana, and were entered into a Microsoft Access database. Original paper quizzes had no participant-identifying information and had been labeled with a participant identification number (ID) and housed in locked warehouses. Data abstracted included variables present on the comprehension quiz: participant ID; day, month, and year that the quiz was administered; answers the participant gave for each of the 20 true/false quiz items; and language of administration (English or Setswana).
Quality assurance was carried out for the quizzes from a randomly selected 12% of the participant IDs following data collection. The entry of their quizzes into the Access database was checked for accuracy against the original paper quizzes and inconsistencies were corrected.
Electronic data files were transported to the Johns Hopkins Berman Institute of Bioethics for analysis. The Access database was converted to a STATA file using Stat Transfer. Data were analyzed using STATA 10.
Additional data, including demographic variables of study participants, were obtained from the study team and linked with quiz data. Subjects whose enrollment date did not match the date of the enrollment quiz in the original dataset within a window of 30 days were eliminated from the dataset.
A quiz number (1–6) was assigned to the quizzes for each participant ID, with the enrollment quiz always being quiz 1. An indicator was created for participants who had 2 baseline quizzes, as this signified that the participant was reported to have failed the first enrollment quiz and was therefore given the enrollment quiz twice. An indicator was created for each question so that we could identify which questions were answered correctly, incorrectly, or left blank, and an additional variable was created to sum the number of questions answered correctly. Variables were created to determine the time since the enrollment quiz and the time since the previous quiz.
To ensure accurate interpretation of quiz items we asked 8 individuals who were unrelated to the trial and who were fluent in both English and Setswana to independently review and back-translate 4 of the Setswana quiz questions that were suspected to have been incorrectly translated into Setswana after the quiz analysis began.
Bivariate and adjusted logistic regression analyses were performed for the enrollment quizzes to determine associations between demographic features and rates of passing at enrollment, as well as the likelihood of answering specific questions correctly at enrollment. These logistic regression analyses were also performed for the entire longitudinal dataset.
Bivariate analyses were performed to determine the odds of passing each follow up quiz compared with the enrollment quiz and compared with the previous quiz, as well as the odds of passing a quiz by the interval since the last quiz.
The Botswana IPT Trial was approved by the Botswana Ministry of Health ethics committee and the CDC institutional review board. This secondary analysis of comprehension quiz data was approved by the Johns Hopkins Bloomberg School of Public Health IRB and the Botswana ethics committee and CDC institutional review board. The trial was registered at clinicaltrials.gov (NCT00164281).
From 2004 to 2006, 4,018 individuals agreed to screening for the IPT trial. Of 1,860 individuals deemed ineligible for participation, 25 were ineligible because they failed the comprehension quiz twice. A total of 1,995 participants were enrolled in the IPT trial
Freq. | % | |
Enrollment | 1,835 | 33 |
2nd attempt, enrollment | 96 | 2 |
Quiz 2 | 1,436 | 26 |
Quiz 3 | 1,306 | 24 |
Quiz 4 | 853 | 15 |
Quiz 5 | 28 | <1 |
Quiz 6 | 1 | <1 |
|
|
|
Comprehension quizzes were administered at screening for enrolment (Quiz 1) and then every 6 months for a period of 36 months. However because not all participants were administered all quizzes, “Quiz 2” – for instance – refers to the second quiz taken regardless of the time elapsed since Quiz 1.
A total of 495 randomly selected quizzes for 233 participants were checked against the original paper quizzes. Six data entry errors were detected, resulting in an error rate of less than 0.05%.
The demographic characteristics of the study population are displayed in
Characteristic | N | % |
|
||
Female | 1,318 | 72 |
Male | 516 | 28 |
Missing | 1 | <1 |
|
||
<20 | 8 | 0.4 |
20–29 | 586 | 32 |
30–39 | 833 | 45 |
40–49 | 337 | 18 |
50–59 | 63 | 3 |
60–69 | 6 | <1 |
70+ | 1 | <1 |
Missing | 1 | <1 |
|
||
Student | 33 | 2 |
Administrator, Professional | 156 | 8 |
Clerk, Technician | 108 | 6 |
Service Worker, Sales | 301 | 16 |
Crafts, Trades, Assemblers | 107 | 6 |
Skilled Agricultural | 5 | 0.3 |
Laborer, Miner, Construction, Transport | 420 | 23 |
Unemployed | 581 | 32 |
Other | 122 | 7 |
Missing | 2 | <1 |
|
||
None | 119 | 6 |
Primary | 511 | 28 |
Secondary | 1,033 | 56 |
Tertiary | 164 | 9 |
Other | 7 | <1 |
Missing | 1 | <1 |
At enrollment, 47% of quizzes were taken in Setswana, although longitudinally 66% of all quizzes were taken in Setswana. Participants did not remain consistent in their choice of language for quiz administration; of the participants who had more than 1 quiz available for analysis, 10% took all of their quizzes in English and 37% took all of their quizzes in Setswana. More than half switched their quiz language preference throughout the trial.
The final data set included enrollment quizzes from 1,835 participants. The overall median score at enrollment was 17 correct responses from among 20 true/false questions. At enrollment, 1,475 participants (80%) passed on the first attempt, and 342 (19%) failed initially but then passed on their second attempt (the remainder did not have a score recorded). The median score for those who failed on their first attempt was 15.
There was a decrease in the proportion of participants who received a passing score for quiz 2 (63%). The odds of passing quiz 2 were significantly lower compared to quiz 1 (OR = 0.4 [0.4, 0.5]). The proportion of participants passing the comprehension quizzes rose following quiz 2; however the odds of passing remained lower than those of passing quiz 1 for all except quiz 5, for which the odds was not statistically significantly different than quiz 1 (
Quiz number | |||||
Quiz 1 | Quiz 2 | Quiz 3 | Quiz 4 | Quiz 5 | |
n = 1817 | n = 1418 | n = 1298 | n = 848 | n = 28 | |
Proportion passing | 80% | 63% | 72% | 76% | 86% |
OR (95% CI) compared to Quiz 1 | 0.4 (0.4, 0.5) | 0.6 (0.5, 0.7) | 0.7 (0.6, 0.9) | 1.4 (0.5, 4.0) | |
OR (95% CI) compared to previous quiz | 0.4 (0.4, 0.5) | 1.5 (1.2, 1.7) | 1.2 (1.0, 1.5) | 1.9 (0.6, 5.5) |
Months since last quiz | |||||
1–6 | 7–12 | 13–18 | 19–24 | >24 | |
n = 166 | n = 683 | n = 440 | n = 562 | n = 1742 | |
Proportion passing | 69% | 65% | 61% | 69% | 75% |
OR (95% CI) compared to 1–6 months | 0.8 (0.6, 1.2) | 0.7 (0.5, 1.1) | 1.0 (0.7, 1.5) | 1.3 (0.9, 1.9) |
The proportion and odds of passing a repeat comprehension quiz by time since the last quiz (months) are displayed in
Higher rates of passing were significantly associated with higher educational attainment.
Highest level of education |
||||
None | Primary | Secondary | Tertiary | |
n = 117 | n = 509 | n = 1021 | n = 162 | |
Proportion passing at enrollment | 73% | 76% | 83% | 89% |
OR (95% CI) compared to no education | 1.2 (0.8, 1.9) | 1.9 (1.2, 2.9) | 3.0 (1.6, 5.7) | |
Proportion taking enrollment quiz in English |
43% | 43% | 54% | 78% |
OR (95% CI) taking quiz in English compared to no formal education | 1.0 (0.7, 1.5) | 1.6 (1.1, 2.4) | 4.9 (2.9, 8.2) | |
OR (95% CI) taking quiz in English compared to previous level | 1.0 (0.7, 1.5) | 1.6 (1.3, 2.0) | 3.0 (2.0, 4.5) |
Level of education coded as “Other” not included.
Only records with score recorded included.
Participants with higher education were more likely to take the quizzes in English than those with lower education (
OR Passing (95% CI) | ||
Unadjusted | Adjusted | |
English | 3.3 (2.5, 4.2) | 3.1 (2.4, 4.0) |
>primary education | 1.7 (1.4, 2.2) | 1.5 (1.2, 1.9) |
English | 1.5 (1.1, 1.9) | 1.3 (1.0, 1.7) |
>primary education | 1.5 (1.2, 1.8) | 1.4 (1.1, 1.7) |
English | 1.9 (1.4, 2.6) | 1.4 (1.0, 2.0) |
>primary education | 2.4 (1.9, 3.1) | 2.2 (1.7, 2.9) |
English | 2.4 (1.6, 3.6) | 1.4 (1.0, 2.0) |
>primary education | 2.8 (2.0, 3.8) | 2.2 (1.7, 2.9) |
English | ∞ | ∞ |
>primary education | 3 (0.3, 26.2) | 2 (0.2, 17.9) |
Odds ratios (OR) shown compare quiz results of those quizzes taken in English to those quizzes taken in Setswana and compare participants who had greater than primary education to those who had either only primary education or no formal education.
Only participants with a score, language, and level of education included; participants with a level of education coded as “other” not included.
Participants who failed quiz 1 on their initial attempt and then passed on their second attempt (n = 342), were more likely to fail quiz 2 (93/251, OR 0.4, 95%CI0.3–0.5), quiz 3 (81/226, OR 0.4, 95%CI0.3–0.6), and quiz 4 (42/136, OR 0.5, 95%CI0.4–0.8) than participants who passed quiz 1 on their first attempt.
The translation reassessment conducted in the analysis phase revealed discrepancies in how 4 of the 20 questions had been translated. Notably, the eight new translators unanimously agreed that question 9, “By taking the medicine I have a very small chance of having damage to my liver,” was originally translated into Setswana as “This medication will help protect my liver.” This error changed the correct answer from true to false for those who took the quiz in Setswana. Subsequently, all data presented in this manuscript were adjusted to take into account which quiz response should be counted as correct. Additionally, 5 of the 8 translators back-translated the phrase “sugar pill” in questions 2 and 18 as “diabetes pill.” As there was some disagreement among the translators, the data were not adjusted.
Comprehension Question | Category | English (n = 969) | Setswana (n = 866) | OR (95% CI) | ||||
Correct | Incorrect | Blank | Correct | Incorrect | Blank | |||
Q1 “This study is about preventing tuberculosis in people living with HIV/AIDS” | Purpose | 97% | 1% | 2% | 90% | 10% | <1% | 18.0 (7.9, 41.5) |
Q2 “I will know whether I am taking a sugar pill” | Placebo, Blinding | 64% | 34% | 3% | 65% | 35% | <1% | 1.03 (0.85, 1.3) |
Q3 “This study will last for 3 years” | Procedures | 97% | 1% | 2% | 96% | 4% | 0% | 6.03 (2.5, 14.5) |
Q4 “Ten teaspoons of blood will be taken from my arm” | Procedures | 81% | 16% | 4% | 66% | 33% | 1% | 2.6 (2.08, 3.25) |
Q5 “I will have a chest x-ray done” | Procedures | 97% | <1% | 2% | 100% | <1% | 0% | 0.55 (0.1, 2.98) |
Q6 “I must continue taking the medicine even if the medication makes me ill” | Procedures, Voluntariness | 77% | 19% | 3% | 69% | 30% | 1% | 1.72 (1.39, 2.14) |
Q7 “I will no longer receive study medication if I stop taking the medication” | Adherence | 65% | 30% | 5% | 44% | 56% | 1% | 2.79 (2.3, 3.38) |
Q8 “There is almost no chance that someone will find out I am HIV infected” | Risks | 85% | 12% | 3% | 94% | 6% | 0% | 0.43 (0.31, 0.61) |
Q9 “By taking the medicine I have a very small chance of having damage to my liver” | Risks | 81% | 16% | 3% | 45% | 54% | 1% | 6.0 (4.83, 7.46) |
Q10 “I will be seen by a doctor or nurse each time I return for a visit” | Procedures | 96% | 2% | 2% | 98% | 2% | <1% | 0.86 (0.46, 1.6) |
Q11 “I will not receive compensation for transport costs to come to the clinic” | Compensation | 89% | 9% | 2% | 89% | 11% | <1% | 1.15 (0.85, 1.57) |
Q12 “I can refuse to continue in the study at any time” | Voluntariness | 77% | 21% | 2% | 76% | 24% | 0% | 1.12 (0.9, 1.39) |
Q13 “I must give a urine sample at each visit” | Procedures | 77% | 20% | 3% | 86% | 14% | <1% | 0.61 (0.48, 0.79) |
Q14 “My blood will be tested for possible liver damage by the study medication” | Procedures, Risks | 89% | 9% | 2% | 93% | 7% | <1% | 0.71 (0.51, 1.0) |
Q15 “If I become ill with AIDS I will no longer be in the study” | Procedures | 74% | 23% | 3% | 77% | 22% | <1% | 0.96 (0.77, 1.19) |
Q16 “The isoniazid may make me sick by making me feel nauseated” | Risks | 90% | 8% | 2% | 87% | 13% | <1% | 1.61 (1.18, 2.18) |
Q17 “Currently the Botswana government recommends one year of isoniazid preventive treatment for people living with HIV/AIDS” | Background | 87% | 11% | 2% | 84% | 16% | <1% | 1.55 (1.18, 2.03) |
Q18 “A placebo is a sugar pill” | Placebo | 78% | 19% | 3% | 82% | 18% | <1% | 0.89 (0.7, 1.13) |
Q19 “If I think I am hurt by this study I should contact the police station in Gaborone” | Risks | 94% | 4% | 3% | 94% | 6% | 0% | 1.66 (1.08, 2.54) |
Q20 “This study will help people living with HIV/AIDS by finding out if longer preventive treatment with isoniazid will prevent TB” | Purpose | 96% | 1% | 2% | 98% | 2% | 0% | 1.65 (0.84, 3.27) |
Across the entire dataset, questions related to study procedures were most likely to be answered correctly, whereas questions related to randomization, placebos, adherence, or risks were most likely to be answered incorrectly (
Comprehension Question | Category | English (n = 969) | Setswana (n = 866) | OR (95% CI) | ||||
Correct | Incorrect | Blank | Correct | Incorrect | Blank | |||
Q1 “This study is about preventing tuberculosis in people living with HIV/AIDS” | Purpose | 97% | 1% | 2% | 85% | 15% | <1% | 21.81(13.01, 36.54) |
Q2 “I will know whether I am taking a sugar pill” | Placebo, Blinding | 68& | 29% | 3% | 64% | 36% | 1% | 1.28 (1.14, 1.46) |
Q3 “This study will last for 3 years” | Procedures | 97% | 1% | 2% | 96% | 4% | <1% | 3.73 (2.3, 6.05) |
Q4 “Ten teaspoons of blood will be taken from my arm” | Procedures | 80% | 17% | 4% | 71% | 28% | 1% | 1.89 (1.64, 2.18) |
Q5 “I will have a chest x-ray done” | Procedures | 94% | 3% | 3% | 97% | 3% | <1% | 0.91 (0.66, 1.26) |
Q6 “I must continue taking the medicine even if the medication makes me ill” | Procedures, Voluntariness | 80% | 17% | 3% | 75% | 24% | 1% | 1.53 (1.33, 1.76) |
Q7 “I will no longer receive study medication if I stop taking the medication” | Adherence | 57% | 40% | 4% | 54% | 46% | 1% | 1.21 (1.08, 1.36) |
Q8 “There is almost no chance that someone will find out I am HIV infected” | Risks | 82% | 15% | 3% | 90% | 9% | <1% | 0.56 (0.48, 0.67) |
Q9 “By taking the medicine I have a very small chance of having damage to my liver” | Risks | 73% | 25% | 3% | 46% | 53% | 1% | 3.45 (3.05, 3.91) |
Q10 “I will be seen by a doctor or nurse each time I return for a visit” | Procedures | 95% | 3% | 2% | 97% | 3% | <1% | 0.95 (0.69, 1.33) |
Q11 “I will not receive compensation for transport costs to come to the clinic” | Compensation | 90% | 8% | 2% | 91% | 8% | 1% | 0.99 (0.8, 1.21) |
Q12 “I can refuse to continue in the study at any time” | Voluntariness | 75% | 23% | 2% | 74% | 25% | <1% | 1.14 (0.99, 1.29) |
Q13 “I must give a urine sample at each visit” | Procedures | 85% | 12% | 3% | 92% | 8% | <1% | 0.58 (0.49, 0.7) |
Q14 “My blood will be tested for possible liver damage by the study medication” | Procedures, Risks | 91% | 7% | 2% | 95% | 4% | <1% | 0.58 (0.46, 0.74) |
Q15 “If I become ill with AIDS I will no longer be in the study” | Procedures | 80% | 17% | 3% | 87% | 12% | <1% | 0.69 (0.59, 0.8) |
Q16 “The isoniazid may make me sick by making me feel nauseated” | Risks | 77% | 21% | 3% | 79% | 21% | <1% | 0.99 (0.87, 1.14) |
Q17 “Currently the Botswana government recommends one year of isoniazid preventive treatment for people living with HIV/AIDS” | Background | 84% | 14% | 2% | 82% | 18% | <1% | 1.34 (1.15, 1.57) |
Q18 “A placebo is a sugar pill” | Placebo | 75% | 22% | 3% | 66% | 32% | 1% | 1.65 (1.45, 1.88) |
Q19 “If I think I am hurt by this study I should contact the police station in Gaborone” | Risks | 94% | 3% | 3% | 93% | 7% | <1% | 2.25 (1.68, 3.0) |
Q20 “This study will help people living with HIV/AIDS by finding out if longer preventive treatment with isoniazid will prevent TB” | Purpose | 96% | 2% | 2% | 93% | 7% | <1% | 4.46 (3.06, 6.51) |
Comprehension Question | Quiz 2 OR (95% CI) (n = 1417) | Quiz 3 OR (95% CI) (n = 1298) | Quiz 4 OR (95% CI) (n = 848) | Quiz 5 OR (95% CI) (n = 28) |
Q1 “This study is about preventing tuberculosis in people living with HIV/AIDS” | 0.4 (0.3, 0.6) | 0.3 (0.3, 0.5) | 0.3 (0.3, 0.4) | 0.3 (0.1, 1.0) |
Q2 “I will know whether I am taking a sugar pill” | 1.0 (0.9, 1.2) | 1.1 (0.9, 1.2) | 1.1 (0.9, 1.3) | 2.0 (0.8, 4.9) |
Q3 “This study will last for 3 years” | 0.6 (0.4, 0.9) | 0.7 (0.4, 1.0) | 0.8 (0.5, 1.4) | ∞ |
Q4 “Ten teaspoons of blood will be taken from my arm” | 0.9 (0.7, 1.0) | 1.0 (0.8, 1.2) | 1.1 (0.9, 1.4) | 2.7 (0.8, 9.0) |
Q5 “I will have a chest x-ray done” | 0.1 (0.04, 0.2) | 0.1 (0.02, 0.1) | 0.1 (0.02, 0.2) | 0.1 (0.01, 0.8) |
Q6 “I must continue taking the medicine even if the medication makes me ill” | 0.9 (0.8, 1.0) | 1.5 (1.2, 1.7) | 1.8 (1.5, 2.3) | 2.0 (0.7, 5.7) |
Q7 “I will no longer receive study medication if I stop taking the medication” | 0.7 (0.6, 0.8) | 1.0 (0.9, 1.2) | 1.4 (1.2, 1.6) | 0.9 (0.4, 1.9) |
Q8 “There is almost no chance that someone will find out I am HIV infected” | 0.7 (0.6, 0.9) | 0.7 (0.6, 0.9) | 0.8 (0.6, 1.1) | 0.9 (0.3, 3.0) |
Q9 “By taking the medicine I have a very small chance of having damage to my liver” | 0.5 (0.4, 0.6) | 0.6 (0.5, 0.7) | 0.6 (0.5, 0.8) | 0.7 (0.3, 1.5) |
Q10 “I will be seen by a doctor or nurse each time I return for a visit” | 0.6 (0.4, 0.9) | 0.7 (0.4, 1.0) | 0.9 (0.5, 1.5) | ∞ |
Q11 “I will not receive compensation for transport costs to come to the clinic” | 1.3 (1.0, 1.7) | 1.4 (1.1, 1.9) | 1.8 (1.3, 2.5) | ∞ |
Q12 “I can refuse to continue in the study at any time” | 0.6 (0.5, 0.7) | 0.9 (0.8, 1.1) | 1.2 (0.9, 1.4) | 0.9 (0.4, 2.1) |
Q13 “I must give a urine sample at each visit” | 3.4 (2.6, 4.4) | 4.3 (3.2, 5.7) | 3.8 (2.7, 5.2) | 2.8 (0.7, 11.7) |
Q14 “My blood will be tested for possible liver damage by the study medication” | 1.7 (1.3, 2.3) | 2.7 (1.9, 3.8) | 3.6 (2.2, 5.7) | ∞ |
Q15 “If I become ill with AIDS I will no longer be in the study” | 2.5 (2.1, 3.1) | 2.7 (2.2, 3.3) | 3.6 (2.7, 4.8) | 8.0 (1.1, 59.0) |
Q16 “The isoniazid may make me sick by making me feel nauseated” | 0.3 (0.2, 0.3) | 0.4 (0.3, 0.4) | 0.4 (0.3, 0.5) | 0.4 (0.2, 1.1) |
Q17 “Currently the Botswana government recommends one year of isoniazid preventive treatment for people living with HIV/AIDS” | 0.7 (0.6, 0.8) | 0.7 (0.6, 0.9) | 0.8 (0.6, 1.0) | 0.9 (0.3, 2.7) |
Q18 “A placebo is a sugar pill” | 0.4 (0.3, 0.5) | 0.5 (0.4, 0.5) | 0.5 (0.4, 0.6) | 0.5 (0.2, 1.1) |
Q19 “If I think I am hurt by this study I should contact the police station in Gaborone” | 0.8 (0.6, 1.0) | 1.0 (0.7, 1.3) | 0.9 (0.6, 1.2) | ∞ |
Q20 “This study will help people living with HIV/AIDS by finding out if longer preventive treatment with isoniazid will prevent TB” | 0.3 (0.2, 0.5) | 0.3 (0.2, 0.4) | 0.2 (0.1, 0.3) | 0.3 (0.1, 1.1) |
This study measured comprehension among 1,835 adults in Botswana recruited for a longitudinal TB prevention trial through repeated administration of a quiz to test participants' comprehension of key study concepts. Participants who had a higher education or took the quizzes in English rather than Setswana demonstrated better comprehension. We also observed that repetition of the quiz during the 2-year period improved comprehension and that a significant proportion of participants had difficulty responding correctly to questions about placebo, randomization, and adverse events.
Having had more years of education was associated with higher rates of passing, both at enrollment and during follow-up. These data reinforce findings of many other studies that higher levels of education are associated with higher levels of understanding of medical research
Examination of specific quiz items revealed that participants tended to answer questions correctly that related to study purpose and procedures, while some questions related to randomization, placebos, risks, and voluntariness were answered correctly less than 70% of the time. While it is troubling that these particular issues were problematic with respect to informed consent, it is consistent with literature demonstrating that these items are particularly likely to be misunderstood among subjects in both the developed and developing world
Various investigators have tested mechanisms to improve how study information is delivered to participants with mixed results
Quiz 2 was administered at 6 months after enrolment to a minority of participants: the range in the time since the enrolment quiz when quiz 2 was administered was broad. This heterogeneous timing of quiz 2's administration combined scores of participants who had taken the quiz 6 months previously with some who had received it years before and may have forgotten certain elements. In addition to this limitation, the observation that repeated quizzes improved comprehension may represent survivor bias: those who understood the study requirements may have been more likely to remain enrolled than those who did not. A further word of caution is that since there were only 20 comprehension questions, missing just two questions resulted in a 10% lower score, and although such a drop may have been determined to be a statistically significant decline, it may have not been meaningful with respect to actual comprehension. Future work might determine which subsets of quiz questions were considered by investigators to be particularly critical to participant understanding and examine trends in correctly answering these questions over time.
Back-translation of several items from the Setswana version of the quiz revealed discrepancies between the Setswana and English versions. While we adjusted what counted as a correct answer to correspond to the accurate meaning of one particular question, this likely introduced some confusion, particularly for the 53% who took the quiz in different languages on different study visits. Confusion may have been most acute for the items related to placebo. Specifically, 5 of 8 of our post-study translators reported that the phrase “sugar pill” from the Setswana quiz could be understood colloquially as “diabetes pill” because, according to the translators, in vernacular Setswana the term for “sugar” and “diabetes” is medically the same
This discrepancy highlights the challenge of creating consent documents and other study materials in multiple languages. Presenting study information clearly is difficult in one's native language; accurately translating what are sometimes abstract or unusual concepts (such as randomization) has been consistently documented as problematic in international research
True/false quizzes are an imperfect means of measuring understanding, as they allow participants to guess answers as well as to repeat back what they heard, whether or not they understood it. Research has demonstrated that asking participants to voice, in their own words, their understanding of research more thoroughly reveals what subjects understand
This study demonstrated that participants in a large 3-year clinical trial in Africa generally understood key study information. Importantly, it also demonstrated that administration of a quiz both at enrollment and follow-up was feasible and served as a useful means of determining whether subjects had sufficient information to enroll in the trial. Participants' understanding of information decreased slightly following enrollment, but the rate of passing improved following the first re-assessment, suggesting that providing quizzes over the course of an ongoing clinical trial may reinforce key study information. Assessments of understanding should be incorporated into future research, particularly in relation to topics that may be less familiar to research participants.