https://orcid.org/0000-0001-9982-6467
Figures
Abstract
Some studies have found a bilingual advantage in children’s executive function and some failed to find a bilingual advantage. For example, the results of a previous study by Bialystok & Martin (2004) indicated that Chinese-English bilingual preschool children outperformed English monolingual children in solving the dimensional change card sort (DCCS). The goal of our study was to replicate this study using the same dimensional change card sort task. We also tested our participants on vocabulary and digit span. Our participants were 40 English monolingual and 40 Mandarin-English bilingual children and were within the same age range as the children in Bialystok & Martin’s (2004) study. Our results showed no difference between bilinguals and monolinguals. Both groups of children in the present study performed better than those in Bialystok and Martin (2004), but the bigger difference was between the two groups of monolinguals. These results suggest that it could be important to attend to monolingual children’s performance, in addition to bilinguals’, when testing for a bilingual advantage. Our replication study is important because it helps with clarifying the validity of studies finding a bilingual advantage and to help future researchers know whether to build on their findings or not.
Citation: Shokrkon A, Nicoladis E (2021) Absence of a bilingual cognitive flexibility advantage: A replication study in preschoolers. PLoS ONE 16(8): e0255157. https://doi.org/10.1371/journal.pone.0255157
Editor: Koji Miwa, Nagoya University, JAPAN
Received: February 3, 2021; Accepted: July 8, 2021; Published: August 5, 2021
Copyright: © 2021 Shokrkon, Nicoladis. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The data set is now published and is downloadable in figshare. https://figshare.com/articles/dataset/absence_of_a_bilingual_advantage_in_cognitive_flexibilty/14530005.
Funding: This study received funding from a Discovery Grant (#2018-04978) from the Natural Sciences and Engineering Research Council of Canada to the second author.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Many studies have found that bilinguals have enhanced cognitive abilities relative to monolinguals [1–6]. Studies have found bilingual advantages on spatial problems [7], mental flexibility [8], metacognitive skills [9], learning strategies [10], and executive function (EF). EF is a set of cognitive processes that regulate one’s thoughts and enable engagement in goal-directed behaviors [11, 12]. Although there are many conceptualizations of EF components, there is general agreement that EF consists of three main components, attention and inhibition (the ability to control attention, behavior, and thoughts), working memory (a mental workplace in which information is held temporarily and mentally manipulated) and cognitive flexibility (allows us to think divergently, change perspective and adapt to a constantly changing environment) [13]. Bilingual advantages have been found in all three aspects of EF: inhibition [5, 14–18], shifting [6, 19–21], and working memory [16, 22, 23]. The bilingual advantage has been attributed to bilinguals’ greater practice in inhibitory processes and attentional control in processing their language(s), such as inhibiting the non-target language [24] or actively attending to the target language [25].
However, many studies have failed to find a bilingual advantage on cognitive measures [26–30]. The lack of bilingual advantage has also been found in all three domains of EF: inhibition [28, 31–33] shifting [28, 31], and working memory [34–36]. Researchers have proposed a number of explanations for these discrepant findings, including physical activity and dietary intake [37, 38], culture [39, 40], the age of participants [27], demographic factors [29, 33, 41], weak psychometric properties of cognitive tasks [28, 41], sample size [42], and publication bias [43].
Since there are somewhat contradictory results in the field of bilingualism and EF, to explore whether speaking more than one language actually impacts the development of EF, one enlightening step is replication [44–47]. As the recent replication crisis gave rise to the importance of replication in order to evaluate whether published results reflect true findings or false positives, the purpose of the present study is to replicate a highly cited study by Bialystok and Martin (2004) showing a bilingual advantage [19] with a very similar population from the same country and using the same measures. This approach is also justified in light of the limited reproducibility of research in the field of psychology [48].
In their study, Bialystok and Martin (2004) compared 36 English monolinguals (18 boys and 18 girls, M = 59.1 months) and 31 Chinese-English bilinguals (21 boys and 10 girls, M = 58.9 months) [19]. They measured the children’s receptive vocabulary in English (with the Peabody Picture Vocabulary Test) and verified that the children did not differ on verbal working memory capacity (with a forward digit span task). To assess the children’s attention and inhibition component of EF, they administered the dimensional change card sort (DCCS) task. The DCCS is a commonly used measure of EF in preschool children [48]. In this task, children are asked to sort cards with pictures (such as red and blue rabbits and boats) according to one dimension of the pictures (such as color) and, then asked to sort the cards according to another dimension (such as shape). This task requires children to attend selectively to the relevant features of a problem, inhibit attention to irrelevant information, and switch between rules. When asked to switch rules, young children typically perseverate, meaning that they continue to apply the original rule even after the rules have switched [48]. The results of Bialystok and Martin (2004) showed that the bilinguals performed better than monolinguals at switching rules. In other words, they found a bilingual advantage [19].
The present study
The main objective of the present study is to replicate previous work by Bialystok and Martin (2004), using the same tasks and with a very similar population. In this study, we compared Mandarin Chinese-English bilingual children’s performance on the DCCS to that of English monolingual children. The children were within the same age range as the children in Bialystok and Martin (2004) [19]. We expected to replicate their results and find a bilingual advantage in attention and inhibition component of EF.
Materials and methods
Participants
80 children were included in the analyses: 40 Mandarin Chinese-English bilingual children (16 girls and 24 boys) and 40 English monolingual children (23 girls and 17 boys), picking the closest matching on age to the bilinguals from a database of 79 English monolinguals. Both the monolinguals and the bilinguals were living in Edmonton, Alberta, Canada. The bilinguals averaged 63.5 months of age (SD = 9.4; range: 48–80) and the monolinguals averaged 62.0 month (SD = 7.4, range: 47–83). The bilingual children were sequential bilinguals, having first learned Mandarin at home with their families. They started to learn English between the ages of two and four years, usually when they started daycare or preschool.
In spite of the fact that the data was originally collected for another study and we did not have a direct measure of socioeconomic status (SES) of the children’s families, we had attempted to control for SES through our recruitment methods. The monolingual participants were recruited from daycares close to the university that are known to have many children of academics. The bilingual participants were all children of graduate students and postdoctoral fellows. Thus, most of our participants were likely from high SES families. We discuss possible effects of SES in the Discussion.
Procedure
This study was reviewed for its adherence to ethical guidelines by a Research Ethics Board at the University of Alberta (Pro00040247). Parents gave us permission to test their children by signing consent forms. Also, verbal assent was asked from the children before testing. The order of the tests within a testing session depended on each child and the experimenter, but we started with more passive tasks, such as the receptive vocabulary test and then continued with the Dimensional Change Card Sort task.
Measures
Vocabulary.
The Peabody Picture Vocabulary Test III (PPVT) was used to measure receptive vocabulary size [49]. In the present study, we report the children’s raw scores.
Working memory.
The Wechsler Intelligence Scale for Children (WISC-R) forward digit span subtest was used to measure verbal working memory [50]. In this test, a random string of digits was presented to children and they were asked to repeat the exact strings of digits. The test started with one digit. If the child repeated back the digit correctly then another digit was added to the string to be remembered. After one error, the test ended. The score represents the highest number of digits the child repeated without errors. The possible range of scores for this measure is 0–8; the actual range of scores was 2–7.
Executive function measure.
We used the same Dimensional Change Card Sort (DCCS) as Bialystok and Martin (2004) for children in the same age range as the present study [19]. For this task, children were asked to sort a set of 10 cards into two groups based on one characteristic of the stimuli and then to resort the same set of 10 cards based on a different characteristic. The task was comprised of the following four conditions or games:
- Colour game. In this condition, the stimuli were five red squares and five blue squares. The children were first told to sort the cards according to their colour. So, the experimenter told them: “This is the color game. In this game, the red ones go here and the blue ones go here” After 10 trials, children were told that they were going to play a new game with new rules. They were told: “In this game, the red ones go here (in the place they used to put blue cards) and the blue ones go there (in the place they used to put red cards).”
- Color-shape game. The stimuli were blue squares and red circles. In the pre-switch phase, there were two boxes with a target stimulus on them. One box had a picture of a red square and the other box had a picture of a blue square. In the pre-switch game, the children were told to place all the blue cards in the box with the picture of the blue square and all the red cards in the box with the picture of the red square. In the post-switch phase, children were told to put all the square cards in the box with the picture of a square and all the circle cards in the box with the picture of a circle.
- Color-object game. This game was the same as the previous game except that meaningful objects were used. The stimuli were red flowers and blue rabbits and the two boxes had the pictures of a red rabbit and a blue flower.
- Function-location game. In this game, instead of perceptual characteristics, the sorting dimensions were abstract features of the stimuli (function vs. location). The set included five things to play with that went outside the house (bicycle, skateboard, pail and shovel, skipping rope, kite) and five things to wear that went inside the house (slippers, nightgown, bib, ballet shoes, and baby pyjamas). On the sorting boxes, there were pictures of a teddy bear (play-inside) and a winter jacket (wear-outside). In the location game, the children were told to put the things that go inside the house in the box with the picture of the teddy bear and the thing that goes outside the house in the box with the picture of the jacket. In the function game, children were told to put the thing to play with inside the box with the picture of the teddy bear and the things to wear in the box with the picture of the jacket.
The first two conditions of this DCCS are similar to the standard DCCS described by Zelazo (2006) [48]. By the age of five years, almost all children can switch rules easily for the colour and shape games [48, 51]. Indeed, one study found that 92% of four-year olds could successfully switch rules for a DCCS similar to the first two conditions [52]. As far as we know, the latter two conditions are unique to Bialystok and Martin (2004) [19]. There was one difference between the DCCS task we used and that of Bialystok and Martin (2004): we administered it on paper and they administered it on a computer. Bialystok and Martin (2004) asserted that the computer administration made the task age-appropriate [19]. We keep in mind that possibility as we interpret the results.
Results
The bilinguals scored significantly lower (M = 33.5, SD = 19.6) than the monolinguals (M = 89.3, SD = 26.4) on the PPVT, t(77) = -10.53, p < .001. The bilinguals did not differ on the forward digit span (M = 4.5, SD = 1.0) from the monolinguals, t (M = 4.4, SD = 0.8), t(78) = .73, p = .47).
Then we examined the scores in a three-way ANOVA for game (4), pre/post (2) and language group (2), with the first two variables as repeated measures, as Bialystok and Martin (2004) did [19]. The results showed only a main effect of game, F (3, 117) = 60.05, p < .001. Post-hoc contrast analyses showed that the fourth game (the function-location game) was the most difficult game, while the other three games were not different from each other. The main effect results of pre/post, F (1, 39) = 3.83, p = .14, and language groups, F (1, 39) = 2.40, p = .12. The only interaction to reach significance was between game and phase, F (3, 117) = 2.74, p = .03. This interaction is likely due to the children’s worse performance on the fourth game following the rule change than before the rule change, while there was little change for the other four games (see Fig 1).
Figs 1 and 2 summarize the children’s number correct (out of 10) at each level of difficulty both before switching rules (Fig 1) and after switching rules (Fig 2).
As Bialystok and Martin (2004) did [19], we next divided the children in our study into perseverators (0–3 on post trials), guessers (4–6) and correct (7–10). We then performed chi-squares comparing the distribution between bilinguals and monolinguals (see Table 1). There were no significant differences between groups.
We next compared the distribution in our study with that of Bialystok and Martin’s (2004) [19]. The results for the bilinguals are summarized in Table 2, for the monolinguals in Table 3. As can be seen in Table 2, our bilinguals scored higher than theirs for the first three levels of the DCCS, but not the fourth. However, as can be seen in Table 3, our monolinguals scored much higher than the monolinguals in Bialystok and Martin (2004), significantly higher on all four levels of the DCCS [19].
Finally, Table 4 summarizes the correlations between post-switch game 4 and age, vocabulary, and forward digit span. For the other three games, there was not enough variability to perform correlations. As can be seen in Table 4, forward digit span is the only significant predictor for both bilinguals and monolinguals. In addition, age and vocabulary are significantly correlated with only bilinguals’ performance.
Discussion
The purpose of the present study was to replicate Bialystok and Martin (2004). As the inconsistencies around the hypothesis of “bilingual advantage” have risen in recent years, it is important to replicate the results of the existing studies to make sure about the validity of studies and to help future researchers know whether to build on their findings or not.
Our participants were within the same age range as Bialystok and Martin (2004), averaging just a few months older and we had a slightly larger sample size. Our participants were from the same country as Bialystok and Martin (2004) and we used the same dimensional change card sort (DCCS) task as they did, with the only difference that we used the paper version of the test rather than the computer version.
Our results showed no bilingual advantage on the DCCS in this study (as have other studies; 28, 31–33]. Furthermore, on the DCCS task, while our bilinguals were slightly better than the bilinguals in Bialystok and Martin (2004), our monolinguals performed much better than theirs. It is possible that the computerized version of the DCCS was harder for the children (as claimed by Bialystok & Martin, 2004). If so, the greater difficulty might account for the difference between the two bilingual groups. However, the difference between the two monolingual groups was much larger, suggesting that there was some other factor(s) at play. Moreover, the monolinguals in our study performed at ceiling on the first levels of the DCCS, as has been reported in similar studies with monolinguals of this age [49, 51].
We speculate that there could be multiple factors responsible for our monolinguals doing better than their monolinguals. First, Paap (2014) showed that most studies showing a bilingual advantage were small-scale studies (average n = 29) [53]. The small number of participants may reduce statistical power, and false-positive findings are more commonly found in small sample studies [54, 55]. Many studies with large sample sizes have not shown a bilingual advantage [32, 56, 57]. Neither our study nor that of Bialystok and Martin (2004) could be characterized as large, so a future study with a larger sample size could decide which results are more replicable. Second, not all studies showing a bilingual advantage have used measures with strong psychometric properties. For instance, many of these studies have used measures with low levels of convergent and discriminant validity [42]. For instance, using the Flanker task, Poarch and Van hell found that bilinguals performed better than monolingual in inhibitory control [58]. Later, they reanalyzed their data and figured that congruent and incongruent conditions and the difference score across tasks were not correlated, which in turn raises doubt about the convergent validity across tasks [59]. Paap, Anders-Jefferson, Zimiga, Mason, and Mikulinsky (2020) also posited that Flanker, Simon, and spatial Stroop tasks should not be characterized as reflecting inhibitory control [60].
Also, studies testing EF can be compromised by weak test-retest reliability [41]. The DCCS (the first two games in the present study) has been shown to have high test-retest reliability [48], however only for three-year-olds. In the present study, it was not entirely clear that DCCS was tapping EF at all. While the children did show some small declines on the post-test relative to the pre-test, only 14 out of the 80 children (18%) were classified as perseverators. The DCCS is thought to tap EF by showing that children continue to follow the same rule as before the rule change [48]. Children’s post-change performance on the fourth game was strongly correlated with their digit span, suggesting that the DCCS administered here might have been a measure of short-term memory. Finally, another possible reason for our contradicting results with those of Bialystok and Martin (2004) might be related to SES of our participants. Bialystok and Martin (2004) claimed that their monolingual and bilingual participants were from similar SES backgrounds because they lived geographically close to each other [19]. No further information was provided about the participants; therefore, it is not known if the participants lived in well-off or poor neighborhoods. Previous research has indicated that SES is associated with EF [33, 61, 62]. In the present study, we tried to control for SES across the two language groups by recruiting participants who were connected to the university. It is possible that our study included monolinguals from higher SES backgrounds than those in Bialystok and Martin (2004) [19]. Future studies can include individual measures of the families’ SES to test this possibility.
This study has some limitations that need to be considered. First, SES was only semi-controlled as we did not have a direct measure of SES, and instead we attempted to control for SES through our recruitment method. Second, the number of our participants were not much higher than Bialystok and Martin (2004), therefore, a future study with a larger sample size could decide which results are more replicable. Another limitation of our study is that we cannot completely dismiss the possibility of a ceiling effect resulting from our participants being 3 months older than those in Bialystok and Martin (2004) and being from a higher SES family.
Regardless of these limitations, the current study attempts to address the question of whether speaking more than one language actually impacts the development of EF and makes important contributions to the literature suggesting that there might be mediating variables that explain our failure to replicate.
Conclusion
The present study did not replicate the bilingual advantage on executive function performance found in Bialystok and Martin (2004), as both the bilingual and monolingual children in the present study outperformed those in Bialystok and Martin (2004) [19]. However, the monolingual children in the present study performed much better than those in Bialystok and Martin (2004) and closer to the performance of monolinguals from other studies [19]. Thus, it is important to verify that monolingual children perform at an age-appropriate level before concluding that there is a bilingual advantage. Moreover, future researchers in the field of bilingualism and EF, should pay more attention to using age-appropriate and real-life tasks measuring clearly reflected EF components. Also, more attention should be paid to the process of experimental set-up, design, procedure, data collection (e.g., trimming; how outliers are identified and excluded from further analysis), and the choice of statistical analyses which could lead to more variability in research findings [46].
References
- 1. Bialystok E. Cognitive complexity and attentional control in the bilingual mind. Child Development. 1999 May;70(3):636–44.
- 2. Bialystok E. Consequences of bilingualism for cognitive development. Handbook of bilingualism: Psycholinguistic approaches. 2005:417–32.
- 3. Bialystok E, Craik FI, Luk G. Bilingualism: consequences for mind and brain. Trends in Cognitive Sciences. 2012 Apr 1;16(4):240–50. pmid:22464592
- 4. Carlson SM, Meltzoff AN. Bilingual experience and executive functioning in young children. Developmental Science. 2008 Mar;11(2):282–98. pmid:18333982
- 5. Martin-Rhee MM, Bialystok E. The development of two types of inhibitory control in monolingual and bilingual children. Bilingualism. 2008 Mar 1;11(1):81.
- 6. Prior A, MacWhinney B. A bilingual advantage in task switching. Bilingualism. 2010 Apr 1;13(2):253.
- 7. Bialystok E, Majumder S. The relationship between bilingualism and the development of cognitive processes in problem solving. Applied Psycholinguistics. 1998 Jan;19(1):69–85.
- 8.
Cummins J. The Influence of Bilingualism on Cognitive Growth: A Synthesis of Research Findings and Explanatory Hypotheses. Working Papers on Bilingualism, No. 9.
- 9.
Duncan SE. Child bilingualism and cognitive functioning: A study of four Hispanic groups.
- 10. Bochner S. The learning strategies of bilingual versus monolingual students. British Journal of Educational Psychology. 1996 Mar;66(1):83–93.
- 11. Shao Z, Janse E, Visser K, Meyer AS. What do verbal fluency tasks measure? Predictors of verbal fluency performance in older adults. Frontiers in Psychology. 2014 Jul 22;5:772. pmid:25101034
- 12.
Zelazo PD, Carlson SM, Kesek A. The development of executive function in childhood.
- 13. Miyake A, Friedman NP, Emerson MJ, Witzki AH, Howerter A, Wager TD. The unity and diversity of executive functions and their contributions to complex “frontal lobe” tasks: A latent variable analysis. Cognitive Psychology. 2000 Aug 1;41(1):49–100. pmid:10945922
- 14. Bialystok E, Craik FI, Ryan J. Executive control in a modified antisaccade task: Effects of aging and bilingualism. Journal of Experimental Psychology: Learning, Memory, and Cognition. 2006 Nov;32(6):1341. pmid:17087588
- 15. Bialystok E, Craik FI, Klein R, Viswanathan M. Bilingualism, aging, and cognitive control: evidence from the Simon task. Psychology and Aging. 2004 Jun;19(2):290. pmid:15222822
- 16. Bialystok E, Craik F, Luk G. Cognitive control and lexical access in younger and older bilinguals. Journal of Experimental Psychology: Learning, memory, and cognition. 2008 Jul;34(4):859. pmid:18605874
- 17. Kapa LL, Colombo J. Attentional control in early and later bilingual children. Cognitive Development. 2013 Jul 1;28(3):233–46. pmid:24910499
- 18. Emmorey K, Luk G, Pyers JE, Bialystok E. The source of enhanced cognitive control in bilinguals: Evidence from bimodal bilinguals. Psychological Science. 2008 Dec;19(12):1201–6. pmid:19121123
- 19. Bialystok E, Martin MM. Attention and inhibition in bilingual children: Evidence from the dimensional change card sort task. Developmental Science. 2004 Jun;7(3):325–39. pmid:15595373
- 20. Prior A, Gollan TH. Good language-switchers are good task-switchers: Evidence from Spanish–English and Mandarin–English bilinguals. Journal of the International Neuropsychological Society. 2011 May;17(4):682–91. pmid:22882810
- 21. Wiseheart M, Viswanathan M, Bialystok E. Flexibility in task switching by monolinguals and bilinguals. Bilingualism: Language and Cognition. 2016 Jan;19(1):141–6. pmid:26877705
- 22. Blom E, Küntay AC, Messer M, Verhagen J, Leseman P. The benefits of being bilingual: Working memory in bilingual Turkish–Dutch children. Journal of Experimental Child Psychology. 2014 Dec 1;128:105–19. pmid:25160938
- 23. Morales J, Calvo A, Bialystok E. Working memory development in monolingual and bilingual children. Journal of Experimental Child Psychology. 2013 Feb 1;114(2):187–202. pmid:23059128
- 24. Green DW. Mental control of the bilingual lexico-semantic system. Bilingualism: Language and Cognition. 1998 Aug;1(2):67–81.
- 25. Finkbeiner M, Gollan T, Caramazza A. Bilingual lexical access: What’s the (hard) problem. Bilingualism: Language and Cognition. 2006;9(2):153–66.
- 26. Dick AS, Garcia NL, Pruden SM, Thompson WK, Hawes SW, Sutherland MT, et al. No evidence for a bilingual executive function advantage in the ABCD study. Nature Human Behaviour. 2019 Jul;3(7):692–701. pmid:31110341
- 27. Hilchey MD, Klein RM. Are there bilingual advantages on nonlinguistic interference tasks? Implications for the plasticity of executive control processes. Psychonomic bulletin & review. 2011 Aug 1;18(4):625–58. pmid:21674283
- 28. Paap KR, Greenberg ZI. There is no coherent evidence for a bilingual advantage in executive processing. Cognitive Psychology. 2013 Mar 1;66(2):232–58. pmid:23370226
- 29. Paap KR, Johnson HA, Sawi O. Bilingual advantages in executive functioning either do not exist or are restricted to very specific and undetermined circumstances. Cortex. 2015 Aug 1;69:265–78. pmid:26048659
- 30. Zhou B, Krott A. Data trimming procedure can eliminate bilingual cognitive advantage. Psychonomic bulletin & review. 2016 Aug 1;23(4):1221–30. pmid:26608692
- 31. Arizmendi GD, Alt M, Gray S, Hogan TP, Green S, Cowan N. Do bilingual children have an executive function advantage? Results from inhibition, shifting, and updating tasks. Language, Speech, and Hearing Services in Schools. 2018 Jul 5;49(3):356–78. pmid:29978206
- 32. Duñabeitia JA, Hernández JA, Antón E, Macizo P, Estévez A, Fuentes LJ, et al. The inhibitory advantage in bilingual children revisited. Experimental Psychology. 2014. pmid:24217139
- 33. Morton JB, Harper SN. What did Simon say? Revisiting the bilingual advantage. Developmental Science. 2007 Nov;10(6):719–26. pmid:17973787
- 34. Namazi M, Thordardottir E. A working memory, not bilingual advantage, in controlled attention. International Journal of Bilingual Education and Bilingualism. 2010 Sep 1;13(5):597–616.
- 35. Engel de Abreu PM. Working memory in multilingual children: Is there a bilingual effect? Memory. 2011 Jul 1;19(5):529–37. pmid:21864216
- 36. Ratiu I, Azuma T. Working memory capacity: Is there a bilingual advantage? Journal of Cognitive Psychology. 2015 Jan 2;27(1):1–1.
- 37. Best JR. Effects of physical activity on children’s executive function: Contributions of experimental research on aerobic exercise. Developmental review. 2010 Dec 1;30(4):331–51. pmid:21818169
- 38. Kim JY, Kang SW. Relationships between dietary intake and cognitive function in healthy Korean children and adolescents. Journal of lifestyle medicine. 2017 Jan;7(1):10. pmid:28261556
- 39. Samuel S.; Roehr-Brackin K.; Pak H.; Kim H. Cultural effects rather than a bilingual advantage in cognition: A review and an empirical study. Cognit. Sci. 2018, 42, 2313–2341.
- 40. Tran C.D.; Arredondo M.M.; Yoshida H. Differential effects of bilingualism and culture on early attention: A longitudinal study in the U.S., Argentina, and Vietnam. Front. Psychol. 2015, 6, 795. pmid:26150793
- 41. Paap KR, Sawi O. The role of test-retest reliability in measuring individual and group differences in executive functioning. Journal of Neuroscience Methods. 2016 Dec 1;274:81–93. pmid:27720867
- 42. Paap KR, Sawi O. Bilingual advantages in executive functioning: problems in convergent validity, discriminant validity, and the identification of the theoretical constructs. Frontiers in Psychology. 2014 Sep 9;5:962.ref
- 43. De Bruin A, Treccani B, Della Sala S. Cognitive advantage in bilingualism: An example of publication bias? Psychological Science. 2015 Jan;26(1):99–107. pmid:25475825
- 44. Loscalzo J. Irreproducible experimental results: causes, (mis) interpretations, and consequences. Circulation, 125(10): 1211–1214. pmid:22412087
- 45. Roediger HL III. Psychology’s woes and a partial cure: The value of replication. APS Observer. 2012 Jan 31;25(2).
- 46. Poarch GJ, Krott A. A bilingual advantage? An appeal for a change in perspective and recommendations for future research. Behavioral Sciences. 2019 Sep;9(9):95.
- 47. Poarch GJ. Multilingual language control and executive function: a replication study. Frontiers in Communication. 2018 Oct 26;3:46.
- 48. Zelazo PD. The Dimensional Change Card Sort (DCCS): A method of assessing executive function in children. Nature Protocols. 2006 Jun;1(1):297–301. pmid:17406248
- 49. Dunn LM, Dunn LM. PPVT-III: Peabody picture vocabulary test. American Guidance Service; 1997.
- 50.
Wechsler D. Wechsler preschool and primary scale of intelligence—fourth edition. The Psychological Corporation San Antonio, TX. 2012.
- 51. Carlson SM. Developmentally sensitive measures of executive function in preschool children. Developmental Neuropsychology. 2005 Oct 1;28(2):595–616. pmid:16144429
- 52. Kirkham NZ, Cruess L, Diamond A. Helping children apply their knowledge to their behavior on a dimension‐switching task. Developmental Science. 2003 Nov;6(5):449–67.
- 53. Paap KR. The role of componential analysis, categorical hypothesising, replicability and confirmation bias in testing for bilingual advantages in executive functioning. Journal of Cognitive Psychology. 2014 Apr 3;26(3):242–55.
- 54. Paap KR, Johnson HA, Sawi O. Are bilingual advantages dependent upon specific tasks or specific bilingual experiences? Journal of Cognitive Psychology. 2014 Aug 18;26(6):615–39.
- 55. Button KS, Ioannidis JP, Mokrysz C, Nosek BA, Flint J, Robinson ES, et al. Power failure: why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience. 2013 May;14(5):365–76. pmid:23571845
- 56. Antón E, Duñabeitia JA, Estévez A, Hernández JA, Castillo A, Fuentes LJ, et al. Is there a bilingual advantage in the ANT task? Evidence from children. Frontiers in Psychology. 2014 May 7;5:398. pmid:24847298
- 57.
Farah MJ, Noble KG. Socioeconomic Influences on Brain Development: A Preliminary Study. In Mayr U., Awh E., & Keele S. W. (Eds.), Decade of behavior. Developing individuality in the human brain: A tribute to Michael I. Posner. American Psychological Association. 189–208.
- 58. Poarch GJ, Van Hell JG. Executive functions and inhibitory control in multilingual children: Evidence from second-language learners, bilinguals, and trilinguals. Journal of experimental child psychology. 2012 Dec 1;113(4):535–51. pmid:22892367
- 59.
Poarch GJ, Van Hell JG. Does performance on executive function tasks correlate? Evidence from child trilinguals, bilinguals, and second language learners. InBilingualism, Executive Function, and Beyond: Questions and Insights 2019 Jan 1 (pp. 223–236). John Benjamins Publishing Company.
- 60. Paap KR, Anders-Jefferson R, Zimiga B, Mason L, Mikulinsky R. Interference scores have inadequate concurrent and convergent validity: Should we stop using the flanker, Simon, and spatial Stroop tasks? Cognitive research: principles and implications. 2020 Dec;5(1):1–27.
- 61. Last BS, Lawson GM, Breiner K, Steinberg L, Farah MJ. Childhood socioeconomic status and executive function in childhood and beyond. PloS one. 2018 Aug 24;13(8):e0202964. pmid:30142188
- 62. Mezzacappa E. Alerting, orienting, and executive attention: Developmental properties and sociodemographic correlates in an epidemiological sample of young, urban children. Child Development. 2004 Sep;75(5):1373–86. pmid:15369520