The authors have declared that no competing interests exist.
Conceived and designed the experiments: MC WT MB. Performed the experiments: MC WT. Analyzed the data: WT MC AD MS MB. Contributed reagents/materials/analysis tools: WT MC AD MS MB. Wrote the paper: WT MC AD MS MB.
We examined whether academic and professional bachelor students with dyslexia are able to compensate for their spelling deficits with metacognitive experience. Previous research suggested that students with dyslexia may suffer from a dual burden. Not only do they perform worse on spelling but in addition they are not as fully aware of their difficulties as their peers without dyslexia. According to some authors, this is the result of a worse feeling of confidence, which can be considered as a form of metacognition (metacognitive experience). We tried to isolate this metacognitive experience by asking 100 students with dyslexia and 100 matched control students to rate their feeling of confidence in a word spelling task and a proofreading task. Next, we used Signal Detection Analysis to disentangle the effects of proficiency and criterion setting. We found that students with dyslexia showed lower proficiencies but not suboptimal response biases. They were as good at deciding when they could be confident or not as their peers without dyslexia. They just had more cases in which their spelling was wrong. We conclude that the feeling of confidence in our students with dyslexia is as good as in their peers without dyslexia. These findings go against the Dual Burden theory (Krüger & Dunning, 1999), which assumes that people with a skills problem suffer twice as a result of insufficiently developed metacognitive competence. As a result, there is no gain to be expected from extra training of this metacognitive experience in higher education students with dyslexia.
Getting a degree in higher education depends on three groups of variables: adequate intellectual abilities, will to work (achievement motivation), and knowing how to study. The last component, on which we will focus, is usually referred to as metacognition
Metacognitive knowledge refers to beliefs about cognition stored in long term memory. Metacognitive skills deal with the regulation of the cognitive processes needed for good performance. They include (appropriate) effort allocation, time allocation, planning, executing the various steps towards the goal, checking the progress, adapting the modus operandi if necessary, and evaluating the outcome, so that lessons can be learned for future performance. Metacognitive experiences are based on previous involvements with the task at hand (or related learning conditions). They enable the learner to be better aware of the progress made
Much research has confirmed the contribution of metacognition to successful learning e.g.,
Dyslexia is a specific learning disorder characterized by a persistent problem in learning to read and/or write words or in the automatization of the reading and writing process (Dyslexia Foundation Netherlands;
The term dyslexia is no longer used in the DSM-5
There is evidence for a genetic component in dyslexia (e.g.,
Studies on dyslexia and metacognition come to inconclusive results. Kirby, Silvestri, Allingham, Parrila and Lafave
Kruger and Dunning
In a recently published Dutch assessment battery for dyslexia in (young) adults
When considering traditional statistical analyses for metacognition in spelling, we saw ourselves confronted with two challenging statistical issues. First, one has to be careful interpreting significant interaction effects in the presence of main effects
The most elegant way to analyze decision strategies is to make use of signal detection theory, as this nicely separates response criteria from the sensitivity to spelling errors.
Both correct spellings and wrong spellings induce a certain degree of evidence for a potential error. This evidence is not fixed but varies from trial to trial as a function of the noise in the system. As a result, there is a normal distribution of evidence when the spelling is correct and a normal distribution of evidence when the spelling is wrong. For someone with good spelling skills, both distributions are far apart, so that there is virtually no overlap between them. This is estimated by the distance d′. We can expect that d′ will be considerably larger for controls than for dyslexics. The second variable of interest is c, the place where the decision criterion is placed. When c is at the intersection of the two distributions (as in the figure), it is positioned optimally, because then the smallest number of errors overall is made. In that case, c = 0. When c is lower than 0, the participant makes too many error judgments (i.e., does so for too many words with correct spellings). Conversely, when c is higher than 0, the participant fails to see too many wrong spellings. The degree of deviation in c from 0 is therefore a good indication of the dual burden theory.
Recent advances in statistical methods have made signal detection analysis possible for many more designs than before. DeCarlo
According to signal detection theory, both correctly and wrongly spelled words trigger a certain amount of evidence for a spelling error. Because of noise in the system, the degree of evidence triggered by correct and wrong spellings is not always exactly the same, but forms two normal distributions (one for correct spellings and one for wrong spellings). In a proficient speller the degree of error provoked by a correctly spelled word is much lower than the degree of error provoked by a wrongly spelled word. So, it will be easy to make a distinction between both distributions. In contrast, for a poor speller, d′ will be small and there will be a large overlap between the distributions. So, the distance (d′) between the two distributions is an indication of spelling proficiency. This distance is thought to be stable per participant: Individuals cannot manipulate their ability to detect errors.
The second aspect influencing performance is where the response criterion (c) is placed. The total number of judgment errors is minimal when c is placed at the intersection of the two distributions (as shown in
In the present study we wanted to investigate whether Kruger and Dunning's
It is easy to see how the position of c translates into the Dual Burden theory. Not only would students with dyslexia have a smaller value of d′. In addition, they would position the response criteria at a non-optimal place, so that they either fail to recognize the correct spellings they are capable to discriminate (when c<0) or fail to see the spelling errors that are within their reach (when c>0). Deviations of c from 0 indicate that the person has a suboptimal
The study was part of a larger research program in which we also looked at the cognitive profile of the students
Two hundred first-year undergraduate students of higher education participated in this study. All students had graduated from secondary school and were in their first year of a professional bachelor (in colleges for higher education) leading to a professional bachelor degree (after three years of education) or an academic bachelor (in some colleges for higher education and in university, preparing for a master degree) in the surroundings of Ghent (one of the main cities in the Northern, Dutch-speaking half of Belgium). The group consisted of 100 students diagnosed with dyslexia and a control group of 100 students without dyslexia or other known neurological or functional disorders (ADHD, ASD, …). All had normal or corrected-to-normal vision and were native speakers of Dutch. Dutch is a language with a more transparent letter-sound-mapping than English, but less transparent than Spanish or Italian (e.g.,
All students with dyslexia who applied for special facilities at the local support office (vzw Cursief) in the academic year 2009–2010 were asked to participate in the study until we had a total of one hundred. To find a group of 100 participants with dyslexia who completed the full study, we had to approach an initial cohort of some 120 students. Of these 120 students a small number of students chose not to cooperate once the study was explained to them. A few more students were lost because they failed to show up at appointments. The students with dyslexia had been diagnosed prior to our study by trained diagnosticians in accordance with the definition of SDN (Stichting Dyslexie Nederland [Foundation Dyslexia Netherlands],
To reflect the inflow in the first year of higher education as much as possible and to construct homogenous groups, matching criteria for recruitment of the control students were restricted to field of study, gender and age. To recruit the control students we used different methods. We asked the students with dyslexia for several names of fellow classmates who would be interested in participating. Amongst these names we selected someone at random. If the student with dyslexia failed to give names (which was the case for about half of the participants), we recruited them ourselves by means of electronic platforms or the guidance counselors at the institution in question. There was no difference between the two groups in socio-economical level based on the educational level of the mother,
Students without Dyslexia N M ( |
Students with dyslexia N M ( |
Effect size Cohen's |
||
Gender | Male | 46 | 46 | |
Female | 54 | 54 | ||
Studies | University | 66 | 66 | |
College for higher education | 34 | 34 | ||
Age | 19.40 |
19.11 |
NA | |
Fluid IQ | 106.80 |
105.40 |
0.13 | |
Word reading | 100.40 |
77.00 |
1.97* | |
Pseudoword reading | 59.70 |
40.90 |
1.59* | |
Word spelling | 24.60 |
17.50 |
2.05* |
For this study two subtests of the
In the first subtest,
The second subtest,
For each spelling test, there were two scores: (1) the number of correct responses, and (2) the FOC weighted responses. The latter was implemented by the authors of the GL&SCHR as follows for the spelling test. If the students were very certain and they spelled the word correctly, they received five points. If they were almost certain and the word was spelled correctly they received four points. If they were not certain and the spelling was correct, they were given three points. If they were not certain and the spelling was incorrect, they were given two points. If they were almost certain and the spelling was incorrect, they received one point. And finally, if they were very certain about their spelling but it was incorrect, they received zero points.
A similar FOC weighted coding scheme was used for the proofreading task. If the students corrected the error in the target word well and they were very certain, they received five points. If they corrected the target word well but felt less certain, they obtained four points. If they corrected the target word but felt uncertain, they got three points. If they failed to correct the target word or misspelled it while correcting, they got two points if they felt uncertain, one point if the felt almost certain, and zero points if they felt very certain.
The two subtests of the GL&SCHR were part of a larger protocol
Our data fit within the framework of SDT as described in the
Second, we have 200 participants instead of just one participant. This violates the independence assumption of the linear model, so that we have to use a linear mixed effects model, with additional random intercepts and slopes per participant.
Finally, we want to compare students with dyslexia and control subjects. This can be done by adding an extra predictor (Group) to the model. A main effect of Group then indicates that the two groups used different response criteria. An interaction of Group with Type of trial (the word is spelled correctly or not) points to a difference in the ability to discriminate between trials with and without errors.
The analyses were based on the clmm function from the R package Ordinal
The rating had three levels (certainly correct, rather certain correct, uncertain), so we had an intercept for the transition between certainly correct and rather certain correct, and a second intercept for the transition between rather certain correct and uncertain. Trial Type was contrast coded: the variable was set to −0.5 for ‘correct’ trials and +0.5 for ‘wrong’ trials. By doing this, the intercepts (response criteria) are counted relative to the point where the ‘correct’ and ‘wrong’ distributions cross (i.e., where c = 0).
First, we analyzed the scores as recommended by the authors of the test. For each test, we had two scores: (1) the number of correct responses, and (2) the FOC weighted responses. Assuming that students with dyslexia are poorer in metacognition than their peers, we expected the difference between both groups to be larger for the FOC weighted scores. As can be seen in
Students with dyslexia | Students without dyslexia | Cohen's |
||||||
M | M | lower CI | upper CI | |||||
Word Spelling | ||||||||
91.59 | 121.40 | 2.06 | 1.64 | 2.48 | ** | |||
17.49 | 24.60 | 2.05 | 1.63 | 2.47 | ** | |||
Proofreading | ||||||||
50.83 | 62.45 | 0.92 | 0.49 | 1.33 | ** | |||
10.05 | 13.81 | 0.56 | 0.14 | 0.98 | ** |
Traditional analysis of the FOC weighted scores.
For the reasons outlined in the
d′ | c1 | c2 | ||
Word spelling | Control | 1.03 (z = 13.31, p<.001) | 0.12 (z = 3.27, p<.001) | 1.13 (z = 29.15, p<.001) |
Dyslexia | 0.71 (z = 10.54, p<.001) | −0.08 (z = −2.73, p<.001) | 0.93 (z = 27.8, p<.001) | |
0.32 (z = 3.13, p<.001) | −0.20 (z = −4.37, p<.001) | |||
Proofreading | Control | 1.02 (z = 10.35, p<.001) | −0.32 (z = −8.15, p<.001) | 0.94 (z = 22.39, p<.001) |
Dyslexia | 0.84 (z = 8.70, p<.001) | −0.42 (z = −10.76, p<.001) | 0.85 (z = 20.74, p<.001) | |
0.17 (z = 1.29, p = .197) | −0.09 (z = −1.78, p = .074) |
Certainly correct and rather certainly correct, c2 = the criterion between rather certainly correct and uncertain.
The proportional odds assumption claims that the difference for c1 and c2 is identical.
As expected, d′ was larger for the controls than for the students with dyslexia. This simply refers to their (stable) differences in spelling proficiency. In line with
A possible objection against the above interpretation is that, although as a group the students with dyslexia did not perform worse, there were very large individual differences with some individuals with dyslexia having very negative c1-values and others having very positive c1-values. This would be reflected in the random intercepts (i.e., the subject-specific deviations of the response criteria).
Very much the same conclusions apply to the Proofreading task, although here both groups tended to be a bit too fast in their transition from certainly correct to rather certain correct (i.e., they put c1 at a level lower than warranted by their spelling proficiency). More importantly, however, there again was no obvious difference between the control group and the dyslexic group, as would be predicted by the Dual Burden theory.
The present study assessed whether undergraduate students in higher education suffer from bad metacognitive skills, as suggested by the dual burden theory
To examine the issue, we ran two tests with Feeling of Confidence (FOC) judgments. According to the dual burden theory, one would expect the difference between students with dyslexia and controls to be larger for scores that take FOC into account than for scores without FOC. However, we searched for an alternative way of analysis, which would give us much more information about the underlying processes. Signal detection is known to be the best framework to model decisions under uncertainty. Such an analysis allows researchers to distinguish criterion setting (which FOC is) from spelling proficiency. Due to recent statistical developments Signal Detection Analyses have become possible for many more designs than the psychophysical experiments on the basis of which they were originally developed.
The signal detection analysis of our data clearly confirmed the difference in proficiency between both groups (d′) and at the same time told us at which positions participants decided to say they were almost certain of their response or not certain at all anymore. As it turned out, the differences in criterion setting were not that much different between students with dyslexia and controls and most certainly not indicative of less rational strategies in students with dyslexia than in other students. Both groups of students seemed to be very smart in their criterion setting. As a result, we feel not justified to say that students with dyslexia in higher education suffer from a double burden (the situation may be different in primary school; this remains to be examined).
At the same time, the signal detection analysis allows us to evaluate the quality of the tests we used and to suggest improvements. For a start, it does not seem that the two-criterion response (certain, almost certain, uncertain) adds much to a single criterion response (certain: yes/no). In the present study, participants clearly put their most important criterion between certain and almost certain; c1). Only when it was clear that their spelling (or spelling correction) did not make sense, did they use the uncertain alternative.
Second, the dictation task also gave clearer data than the proofreading task. One reason for this may be the fact that participants could zoom in on non-target words in the sentence, thinking these were wrong (although they were not). It might be better in the future to alert the participants to the target words (e.g., by printing them in bold) and ask whether this part of the sentences is correctly spelled: yes or no (and how sure the participants are about their correction: sure vs. not sure). These designs will also be more straightforward to analyze. It would, however, require the addition of sentences in which the target words were spelled correctly.
All in all, we conclude that when signal detection theory analysis is used, there is no evidence for a dual burden in students with dyslexia in higher education. Their metacognitive skills are as good as they can be, given the fact that these students are making more mistakes because of their lower proficiency level. Remediation programs focusing on the metacognitive skill of FOC, therefore, are bound to fail. They may even do more bad than good, if the criterion setting is based on implicit learning rather than explicit, declarative knowledge
Finally, because of the added value of the signal detection analysis and because this analysis may be rather daunting for someone starting with it (it took us quite a bit of time too), we include our data and the R program used to analyze them in the supplementary materials. In that way, everyone can first check whether they know how to do the analyses (by comparing their output with our data to
We are convinced that these new types of analysis, although a little bit more complicated to apply, give a better picture of the true metacognitive qualities of people with more limited cognitive skills. After we finished data collection, we discovered another study of Maniscalco and Lau
(TSV)
(R)