Previous language production research with bidialectals has provided evidence for similar language control processes as during bilingual language production. In the current study, we aim to further investigate this claim by examining bidialectals with a voluntary language switching paradigm. Research with bilinguals performing the voluntary language switching paradigm has consistently shown two effects. First, the cost of switching languages, relative to staying in the same language, is similar across the two languages. The second effect is more uniquely connected to voluntary language switching, namely a benefit when performing in mixed language blocks relative to single language blocks, which has been connected to proactive language control. If a similar pattern could be observed with bidialectals in a voluntary language switching paradigm, then this would provide additional evidence in favor of similar control processes underlying bidialectal and bilingual language production.
Citation: Declerck M, Kirk NW (2021) Is it easier to use one language variety at a time, or mix them? An investigation of voluntary language switching with bidialectals. PLoS ONE 16(9): e0256554. https://doi.org/10.1371/journal.pone.0256554
Editor: Veronica Whitford, University of New Brunswick Fredericton, CANADA
Received: April 1, 2021; Accepted: August 9, 2021; Published: September 8, 2021
Copyright: © 2021 Declerck, Kirk. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data from this study will be made available upon study completion.
Funding: This research is funded by a Carnegie Trust for the Universities of Scotland Research Incentive Grant (https://www.carnegie-trust.org/) awarded to NWK (RIG009864). The funders had and will not have a role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Previous research has indicated that when bidialectals (i.e., speakers of a regional dialect that are also fluent in a standard language variety) produce language, both the standard language and dialect are activated (e.g., [1, 2]), which is assumed to lead to competition among both language varieties. Similar to bilinguals (for reviews, see [3, 4]), a language control process is assumed to be implemented to deal with the competition between language varieties in bidialectals [1, 2, 5, 6]. Some studies have suggested that the language control process implemented by bidialectal speakers of closely related language varieties is similar to the bilingual language control process [1, 2, 5]. In the current study, we set out to further investigate the language control process implemented by bidialectals and its relation to the bilingual language control process by letting bidialectals perform in a voluntary language switching paradigm.
Voluntary language switching [7–16]usually requires participants to name pictures in one of two languages. Unlike other variants of the language switching paradigm (for a review, see ), which indicate the language that should be used for each stimulus through cues (e.g., differently colored frames around the stimuli ), alternating languages (e.g., AABBAABB, with A and B referring to trials in different languages ), or written words (e.g ), voluntary language switching allows the participants to choose the language on each trial.
Similar to other variants of language switching (e.g., [17, 20–23]), voluntary language switching with bilinguals generally results in a cost when switching languages, relative to staying in the same language across trials [8–16] however, see . Indicative of the voluntary language switching paradigm with bilinguals is that these switch costs are similar across languages (i.e., symmetrical switch costs [8–14]). This is in contrast with other variants of the language switching paradigm, where asymmetrical switch costs, which entails larger switch costs in the first language (L1) than in the second language (L2) and is typically used as a measure of inhibitory control (e.g., [17, 24–26]for a review, see ), are relatively often found. According to Gollan and Ferreira , the absence of asymmetrical switch costs with the voluntary language switching paradigm is because this paradigm allows bilinguals to name “easier” words in their L2. This should result in a more similar L1 and L2 activation level for the produced words regardless of language proficiency, and thus might lead to symmetrical switch costs.
Another measure of language control are mixing costs (for a review, see ). Mixing costs entail worse performance in repetition trials in mixed language blocks relative to performance in single language blocks. This measure has been explained with control processes that are implemented in anticipation of any upcoming cross-language competition (i.e., proactive language control; e.g., ) and the mental cost to maintain and monitor two languages (e.g., ). Mixing costs are a highly stable effect in all variants of language switching (e.g., [20, 25, 26, 29–31]), with the exception of voluntary language switching. Voluntary language switching studies actually tend to show a mixing benefit (i.e., worse performance in single language blocks relative to performance in repetition trials in mixed language blocks) in one [11, 12] or both languages [8, 9, 13, 14]. The mixing benefit in L2 observed by Gollan and Ferreira was explained by assuming that only “easier” words are produced in L2 in the voluntary language switching paradigm. Based on this explanation, one would expect that the mixing benefit was only observed for words that were consistently named in L2, but that was not the case . An alternative explanation that can account for a mixing benefit across both languages comes from de Bruin et al. : Production in a single language block requires substantial proactive inhibition of the non-target language [32, 33]. When producing in a mixed language block with voluntary language switching, bilinguals do not require substantial proactive control processes to guide language production, since participants can choose which language to use on any given trial. So, because the implementation of more proactive control processes during single than mixed language blocks is more taxing, better performance is expected in the latter block type when using a voluntary language switching paradigm. From this overview, it appears that voluntary vs. involuntary language switching has a large impact on measures of the bilingual language control process.
In the current study, we set out to investigate if this is also the case for bidialectals by letting bidialectals perform in a voluntary language switching paradigm. The few studies that investigated control processes with bidialectals and bilinguals provide evidence for a shared language control process. Similar to bilinguals, bidialectals show a cost to switching between language varieties, relative to staying in the same language variety across languages [1, 2, 5, 6] Kirk and colleagues , for instance, let bidialectals (English-Orcadian) name pictures during a language switching task, where the language variety on each trail was indicated by differently colored frames corresponding to each language variety (cf. involuntary language switching). The results showed that switching between language varieties results in worse performance than staying in the same language variety across trials. Similar to bilinguals (e.g., ), these bidialectals showed asymmetrical switch costs, with larger switch costs in their more dominant language variety (dialect) than in their less dominant language variety (standard language; see also ). Asymmetrical switch costs were even found with new bidialectals (English-Dundonian), whereas more fluent bidialectals showed symmetrical switch costs . The latter pattern is similar to that observed with second language learners and highly proficient bilinguals, respectively ([21, 34]). Finally, Kirk and colleagues also showed worse performance in repetition trials in mixed language blocks than in single language blocks . So, along the lines of prior bilingual studies (e.g., [26, 29–31]), mixing costs can be observed with bidialectals during involuntary language switching.
The similarities of bidialectals and bilinguals in previous studies that relied on involuntary language switching seem to indicate that similar language control processes are implemented by these two groups during language production. While it might seem obvious that bilinguals and bidialectals rely on the same language control processes, previous related research indicates that is not necessarily the case. For instance, control processes used within the same language has shown to be different to control used between languages . Even more damning for the assumption that similar control processes are used throughout language processing is that some studies found evidence that different language pairs do not necessarily converge when it comes to language control . A similar discrepancy in language control has been observed across modalities within the same bilinguals (e.g., ). These studies provide evidence against the claim that language control is domain general [17, 24], as the control processes within a domain (i.e., language processing) are sometimes even different.
There have also been attempts to objectively distinguish languages from dialects (and thus bilinguals from bidialectals) on a cognitive level using the picture word interference paradigm, which initially suggested dialect items were processed as within-language competitors, akin to synonyms . However, more recent evidence has challenged some of these findings [39, 40]. Thus, the extent to which bidialectals are similar to bilinguals is still unclear and can have theoretical and methodological implications for research comparing bilinguals and monolinguals. For example, research suggesting that there is a general executive control advantage for bilinguals over monolinguals as a result of the regular engagement of language control mechanisms , could be invalidated by the presence of bidialectal speakers who also use these mechanisms, but who are erroneously categorized as monolingual .
To further investigate the issue of whether there are similar language control processes used in bilingual and bidialectal language production, we set out to examine whether a similar pattern can be observed with bidialectals as with bilinguals in a voluntary language switching paradigm. In the current study, we rely on Scottish speakers of a specific type of Scots as the dialect of interest. Although Scots is recognized by the European Charter for Regional or Minority Languages as a separate minority language (from English), it is generally not given this status, with many facing ridicule for suggesting that Scots and English are separate languages . Even a majority of speakers themselves do not hold this view, with one Scottish Government  report demonstrating that 65% of respondents consider their use of Scots as “just a way of speaking”. Consequently, these speakers are likely to identify as monolingual rather than bilingual—or even bidialectal—especially if language background measures are not sensitive to the existence of non-standard varieties (see, [1, 5]).
More specifically, we will test speakers of Dundonian Scots and (Scottish) Standard English. Dundonian is an urban dialect used in and around the Scottish city of Dundee. Like other urban Scots dialects, it exists as a lower status variety in a diglossic situation with (Scottish) Standard English as the prestige variety . Whereas Dundonian Scots overlaps substantially with its corresponding standard language, there are several notable differences. First, it is characterized by phonetic differences relative to the standard language, such as vowel differences (e.g. Standard English ‘pie’ / paɪ / vs. Dundonian ‘peh’ /pε/) and monophthongisation (e.g. Standard English ‘mouse’ /maʊs/ vs. Dundonian ‘moose’ /mu:s/). Second, and most important for the current study, there are also words entirely different in Dundonian than its corresponding translation equivalent in Standard English (e.g., “crying” in Standard English would be “greetin” in Dundonian).
If bidialectals and bilinguals rely on similar language control processes, we expect to observe symmetrical switch costs in the voluntary language switching paradigm with the English-Dundonian bidialectals, similar to the pattern observed with bilinguals. This finding might simply indicate that bidialectals tend to produce “easier” words in the less proficient language . Observing a mixing benefit with English-Dundonian bidialectals in a voluntary language switching paradigm would be a more unique finding, as this effect has only reliably been observed with bilinguals in a voluntary language switching paradigm. A mixing benefit would indicate that bidialectals implement proactive language control in single language blocks, whereas this is less the case in mixed language blocks .
A power analysis on the mixing benefit was run to determine the required number of participants and trials. We chose the mixing benefit, since this effect seems more uniquely connected to voluntary language switching. Along the approach suggested by Brysbaert and Stevens , we ran 200 (Monte Carlo) simulations with the simr package  on the voluntary language switching data of Jevtović et al. , who tested 40 bilinguals with 20 distinct stimuli in 80 trials in single language blocks and 180 trials in voluntary language switching blocks for each participant. The results showed that the setup of Jevtović et al.  had a 99.5% chance of showing a mixing benefit. In the current study, we will rely on the same number of stimuli as Jevtović and colleagues. Furthermore, we will rely on a similar number of participants and trials. The number of voluntary language switching trials per participant will be slightly decreased relative to Jevtović et al. (from 180 to 160 trials per participant). That way, we will have a similar number of trials in the voluntary language switching blocks and the single language blocks. To make sure that we have at least the same number of voluntary language switching block trials across participants as Jevtović and colleagues, we increased the number of participants from 40 to 46.
So, 46 active bidialectal speakers of (Scottish) Standard English and Dundonian Scots will be recruited, all between 18 and 60 years old. These bidialectals will receive the same questionnaire as the bidialectals in Kirk et al. , which will provide us with a subjective measure of the participant’s language proficiency through self-reported proficiency scores, as well as information regarding the interactional context in which each language is used.
The proposed study has received approval from Abertay University’s research ethics committee (EMS4259).
Materials and task
Along the lines of Jevtović et al. , 20 pictures will be presented to the bidialectal participants. These pictures correspond to non-cognate names between Standard English (average number of syllables across the English names: 1.55; average Zipf frequency of the English names: 4.24; ) and Dundonian (average number of syllables across the Dundonian names: 1.60; Zipf frequency was not calculated for Dundonian names as no database with word frequency exists for the Dundonian dialect. See S1 Appendix for the stimulus list). Each picture will be presented twice, in non-consecutive trials, throughout each block.
The experimental task will be made publicly available as “Open Materials” on the Gorilla platform (http://gorilla.sc).
The picture naming study will be presented online on the Gorilla platform . After providing informed consent, participants will perform a microphone check, in which they name a sentence and then listen to their own recording. A familiarization block will follow the microphone check, in which all 20 pictures will be presented together with the corresponding names in Standard English and Dundonian. Because there is no standardized written form of the Dundonian dialect, participants will have the option to listen to a recording of the Dundonian word spoken by a local speaker. The familiarization phase will be followed by the actual experiment, which will consist of two single language blocks of 40 trials each and four voluntary language switching block of 40 trials each. There are several ways to present the order of single language blocks and mixed language blocks to obtain mixing costs or a mixing benefit . We opt for the setup used in the bilingual voluntary language switching studies of de Bruin et al. [8, 9] and Jevtović et al. : Participants will first see one single language block, followed by the four voluntary language switching blocks, and then again one single language block in the other language variety than the first single language block. The language variety of the single language blocks will be counterbalanced across participants.
Prior to each of the three block types (i.e., English language block, Dundonian language block, and the voluntary language switching blocks), instructions will be displayed pertinent for that block type, emphasizing speed and accuracy to name each picture. Moreover, in the single language blocks, the bidialectals will be instructed to name each picture in the corresponding language throughout the block. Prior to the voluntary language switching blocks the following sentences will be presented (for similar instructions, see [8, 14]): “In the following section, you can name the pictures in either Standard English or Dundonian. You are free to choose which language variety to use for each picture. However, please do not use the same language variety throughout the whole task.”. The instructions will be followed by a short demonstration of the task before completing a short practice block consisting of 8 trials. Finally, the participant will perform the experimental block(s).
Each trial will start with a fixation cross in the middle of the screen. After 250 ms, the fixation cross will be replaced by the stimulus for a maximum of 3000 ms. Each trial will end with a brief 50 ms blank screen before the onset of the next trial.
The raw data and analyses scripts will be available on the Open Science Framework.
The first trial in each voluntary language switching block will be excluded from both the error analyses and reaction time (RT) analyses, since this is neither a switch nor repetition trial. Furthermore, error trials and trials immediately following an error will be excluded from the RT analyses. Trials with RTs faster than 150 ms, slower than 3000 ms, or three standard deviations above participant mean will also be removed.
Similar to previous bilingual voluntary language switching studies (e.g., [9, 10]), we will provide an overview of the mean switch rate of the participants in all conditions. Furthermore, two analyses will be conducted on the RT and error data. The switch analyses will be conducted on the data of the voluntary language switching blocks and will consist of Trial type (switch vs. repetition trials) and Language variety (Standard English vs. Dundonian). The mix analyses will be conducted on the single language blocks and the repetition trials in the voluntary language switching blocks. The latter analyses will consist of Block type (repetition trials from voluntary language switching blocks vs. trials from single language block) and Language variety (Standard English vs. Dundonian).
The RTs will be analyzed using linear mixed-effects regression modeling . Furthermore, the error data will be analyzed, if we observe enough errors for meaningful analysis (> 5%), using logistic mixed-effects regression modeling . Both participants and items will be considered random factors with all fixed effects and their interactions varying by all random factors . Yet, convergence issues will be taken into account—we will rely on the buildmer package  to simplify the model where such issues are encountered. For all two-level factors we will be using effect coding (i.e., -0.5 and 0.5). Finally, t- and z-values larger or equal to 1.96 were deemed significant .
- 1. Kirk N. W., Kempe V., Scott-Brown K. C., Philipp A., & Declerck M. (2018). Can monolinguals be like bilinguals? Evidence from dialect switching. Cognition, 170, 164–178. pmid:29024916
- 2. Vorwerg C. C., Suntharam S., & Morand M. A. (2019). Language control and lexical access in diglossic speech production: Evidence from variety switching in speakers of Swiss German. Journal of Memory and Language, 107, 40–53.
- 3. Calabria M., Costa A., Green D. W., & Abutalebi J. (2018). Neural basis of bilingual language control. Annals of the New York Academy of Sciences, 1426, 221–235. pmid:29917244
- 4. Declerck M., & Philipp A. M. (2015). A review of control processes and their locus in language switching. Psychonomic Bulletin & Review, 22, 1630–1645. pmid:25917142
- 5. Kirk N. W., Declerck M., Kemp R., & Kempe V. (2021). Language control in dialect speakers–“monolinguals” by name, bilinguals by nature?. Manuscript submitted for publication.
- 6. Scaltritti M., Peressotti F., & Miozzo M. (2017). Bilingual advantage and language switch: What’s the linkage?. Bilingualism: Language and Cognition, 20, 80–97.
- 7. Blanco-Elorrieta E., & Pylkkänen L. (2017). Bilingual language switching in the laboratory versus in the wild: The spatiotemporal dynamics of adaptive language control. Journal of Neuroscience, 37, 9022–9036. pmid:28821648
- 8. de Bruin A., Samuel A. G., & Duñabeitia J. A. (2018). Voluntary language switching: When and why do bilinguals switch between their languages?. Journal of Memory and Language, 103, 28–43.
- 9. de Bruin A., Samuel A. G., & Duñabeitia J. A. (2020). Examining bilingual language switching across the lifespan in cued and voluntary switching contexts. Journal of Experimental Psychology: Human Perception and Performance, 46, 759–788. pmid:32324035
- 10. Gollan T. H., Kleinman D., & Wierenga C. E. (2014). What’s easier: Doing what you want, or being told what to do? Cued versus voluntary language and task switching. Journal of Experimental Psychology: General, 143, 2167–2195. pmid:25313951
- 11. Gollan T. H., & Ferreira V. S. (2009). Should I stay or should I switch? A cost–benefit analysis of voluntary language switching in young and aging bilinguals. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35, 640–665. pmid:19379041
- 12. Gross M., & Kaushanskaya M. (2015). Voluntary language switching in English–Spanish bilingual children. Journal of Cognitive Psychology, 27, 992–1013. pmid:26889376
- 13. Grunden N., Piazza G., García-Sánchez C., & Calabria M. (2020). Voluntary language switching in the context of bilingual aphasia. Behavioral Sciences, 10, 141. pmid:32962153
- 14. Jevtović M., Duñabeitia J. A., & de Bruin A. (2020). How do bilinguals switch between languages in different interactional contexts? A comparison between voluntary and mandatory language switching. Bilingualism: Language and Cognition, 23, 401–413.
- 15. Kleinman D., & Gollan T. H. (2016). Speaking two languages for the price of one: Bypassing language control mechanisms via accessibility-driven switches. Psychological Science, 27, 700–714. pmid:27016240
- 16. Liu H., Tong J., de Bruin A., Li W., He Y., & Li B. (2020). Is inhibition involved in voluntary language switching? Evidence from transcranial direct current stimulation over the right dorsolateral prefrontal cortex. International Journal of Psychophysiology, 147, 184–192. pmid:31830498
- 17. Meuter R. F., & Allport A. (1999). Bilingual language switching in naming: Asymmetrical costs of language selection. Journal of Memory and Language, 40, 25–40.
- 18. Jackson G. M., Swainson R., Cunnington R., & Jackson S. R. (2001). ERP Correlates of executive control during repeated language-switching. Bilingualism: Language and Cognition, 4, 169–178.
- 19. Macizo P., Bajo T., & Paolieri D. (2012). Language switching and language competition. Second Language Research, 28, 131–149.
- 20. Christoffels I. K., Firk C., & Schiller N. O. (2007). Bilingual language control: An event-related brain potential study. Brain Research, 1147, 192–208. pmid:17391649
- 21. Costa A., & Santesteban M. (2004). Lexical access in bilingual speech production: Evidence from language switching in highly proficient bilinguals and L2 learners. Journal of Memory and Language, 50, 491–511.
- 22. Declerck M., Koch I., & Philipp A. M. (2012). Digits vs. pictures: The influence of stimulus type on language switching. Bilingualism: Language and Cognition, 15, 896–904.
- 23. Declerck M., Koch I., & Philipp A. M. (2015). The minimum requirements of language control: Evidence from sequential predictability effects in language switching. Journal of Experimental Psychology: Learning, Memory, and Cognition, 41, 377–394. pmid:24999708
- 24. Green D. W. (1998). Mental control of the bilingual lexico-semantic system. Bilingualism: Language and Cognition, 1, 213–229.
- 25. Declerck M., Philipp A. M., & Koch I. (2013). Bilingual control: Sequential memory in language switching. Journal of Experimental Psychology: Learning, Memory, and Cognition, 39, 1793. pmid:23773181
- 26. Ma F., Li S., & Guo T. (2016). Reactive and proactive control in bilingual word production: An investigation of influential factors. Journal of Memory and Language, 86, 35–59.
- 27. Bobb S. C., & Wodniecka Z. (2013). Language switching in picture naming: What asymmetric switch costs (do not) tell us about inhibition in bilingual speech planning. Journal of Cognitive Psycholology, 25, 568–585.
- 28. Declerck M. (2020). What about proactive language control?. Psychonomic Bulletin & Review, 27, 24–35. pmid:31410740
- 29. Jylkkä J., Lehtonen M., Lindholm F., Kuusakoski A., & Laine M. (2018). The relationship between general executive functions and bilingual switching and monitoring in language production. Bilingualism: Language and Cognition, 21, 505–522.
- 30. Peeters D., & Dijkstra T. (2018). Sustained inhibition of the native language in bilingual language production: A virtual reality approach. Bilingualism: Language and Cognition, 21, 1035–1061.
- 31. Stasenko A., Matt G. E., & Gollan T. H. (2017). A relative bilingual advantage in switching with preparation: Nuanced explorations of the proposed association between bilingualism and task switching. Journal of Experimental Psychology: General, 146, 1527–1550. pmid:28714710
- 32. Degani T., Kreiner H., Ataria H., & Khateeb F. (2020). The impact of brief exposure to the second language on native language production: Global or item specific? Applied Psycholinguistics, 41, 153–183.
- 33. Wodniecka Z., Szewczyk J., Kałamała P., Mandera P., & Durlik J. (2020). When a second language hits a native language. What ERPs (do and do not) tell us about language retrieval difficulty in bilingual language production. Neuropsychologia, 141, 107390. pmid:32057934
- 34. Costa A., Santesteban M., & Ivanova I. (2006). How do highly proficient bilinguals control their lexicalization process? Inhibitory and language-specific selection mechanisms are both functional. Journal of Experimental Psychology: Learning, Memory, and Cognition, 32, 1057–1074. pmid:16938046
- 35. Declerck M., Ivanova I., Grainger J., & Duñabeitia J. A. (2020). Are similar control processes implemented during single and dual language production? Evidence from switching between speech registers and languages. Bilingualism: Language and Cognition, 23, 694–701.
- 36. Kaufmann E., Mittelberg I., Koch I., & Philipp A. M. (2018). Modality effects in language switching: Evidence for a bimodal advantage. Bilingualism: Language and Cognition, 21, 243–250.
- 37. Blanco-Elorrieta E., & Pylkkänen L. (2016). Bilingual language control in perception versus action: MEG reveals comprehension control mechanisms in anterior cingulate cortex and domain-general control of production in dorsolateral prefrontal cortex. Journal of Neuroscience, 36, 290–301. pmid:26758823
- 38. Melinger A. (2018). Distinguishing languages from dialects: a litmus test using the picture-word interference task. Cognition, 172, 73–88. pmid:29232596
- 39. Melinger A. (2021). Do elevators compete with lifts?: Selecting dialect alternatives. Cognition, 206, 104471. pmid:33075569
- 40. Dylman A. S., & Barry C. (2018). When having two names facilitates lexical selection: Similar results in the picture-word task from translation distractors in bilinguals and synonym distractors in monolinguals. Cognition, 171, 151–171. pmid:29172105
- 41. Bialystok E., Craik F. I., Klein R., & Viswanathan M. (2004). Bilingualism, aging, and cognitive control: evidence from the Simon task. Psychology and aging, 19(2), 290. pmid:15222822
- 42. Kempe V., Kirk N. W., & Brooks P. J. (2015). Revisiting theoretical and causal explanations for the bilingual advantage in executive functioning. Cortex, 73, 342–344. pmid:26298266
- 43. McDermott P. (2019). From ridicule to legitimacy? ‘Contested languages’ and devolved language planning. Current Issues in Language Planning, 20(2), 121–139.
- 44. Scottish Government (2010). Public attitudes towards the Scots language. http://www.gov.scot/Publications/2010/01/06105123/0
- 45. Jauriberry T. (2021). Variation and change of middle-class/r/in Standard Scottish English. Lingua, 103059.
- 46. Brysbaert M., & Stevens M. (2018). Power analysis and effect size in mixed effects models: a tutorial. Journal of Cognition, 1, 9. pmid:31517183
- 47. Green P., & MacLeod C. J. (2016). SIMR: an R package for power analysis of generalized linear mixed models by simulation. Methods in Ecology and Evolution, 7, 493–498.
- 48. van Heuven W. J., Mandera P., Keuleers E., & Brysbaert M. (2014). SUBTLEX-UK: A new and improved word frequency database for British English. Quarterly Journal of Experimental Psychology, 67, 1176–1190. pmid:24417251
- 49. Anwyl-Irvine A. L., Massonnié J., Flitton A., Kirkham N., & Evershed J. K. (2020). Gorilla in our midst: An online behavioral experiment builder. Behavior Research Methods, 52, 388–407. pmid:31016684
- 50. Baayen R. H., Davidson D. J., & Bates D. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59, 390–412.
- 51. Jaeger T. F. (2008). Categorical data analysis: Away from ANOVAs (transformation or not) and towards logit mixed models. Journal of Memory and Language, 59, 434–446. pmid:19884961
- 52. Barr D. J., Levy R., Scheepers C., & Tily H. J. (2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68, 255–278. pmid:24403724
- 53. Voeten, C. C. (2021). Buildmer: Stepwise elimination and term reordering for mixed- effects regression. https://CRAN.R-project.org/package=buildmer.
- 54. Baayen R. (2008). Analyzing Linguistic Data: A practical introduction to statistics. Cambridge University Press.