https://orcid.org/0000-0003-1468-6448
Figures
Abstract
Objective
The Edinburgh Cognitive and Behavioural ALS Screen (ECAS) is an established cognitive screening instrument for patients with amyotrophic lateral sclerosis (ALS). Different from tools like the Mini-Mental State Examination (MMSE), it is adjusted for motor impairment, yet, the latter remains one of the most widely used screening instruments, also in ALS studies. Thus, it is of utmost importance to relate outcome scores of both instruments to allow for comparison in ALS patients. This study reports on the performance of ALS patients in both tests with regard to incidence and degree of cognitive impairment, and the correspondence of both, ECAS and MMSE scores.
Methods
We examined N = 84 ALS patients with the German versions of the ECAS and the MMSE. Performance in both tests regarding incidence and degree of cognitive impairment, and correspondence of frequency of cognitive impairment according to both tests was examined. The relationship between ECAS and MMSE scores was modelled with a non-linear regression model.
Results
All ALS patients were able to complete the ECAS, 89.3% (N = 75) were capable to complete the MMSE. Prevalence of cognitive impairment was in both tests 22.7%, however agreement was only 52.9%. Despite, regression analyses yielded a strong positive relationship (adjusted R2 = .68) between the ECAS total score and the MMSE total score. Both tests were able to identify all patients with dementia.
Conclusion
These results suggest that the MMSE is not ideal for cognitive screening in early-stage ALS patients. However, a rough translation of MMSE scores in ECAS scores is possible to estimate the cognitive performance level of patients, with the ECAS being more discriminative in the lower range of cognitive dysfunction (ECAS score: 80–136), for which the MMSE does not define cognitive impairment (corresponding MMSE score: 27–30).
Citation: Serian A, Finsel J, Ludolph AC, Uttner I, Lulé D (2024) Screening instruments of cognition: The relation of the mini-mental state examination to the Edinburgh cognitive and behavioural ALS screen in amyotrophic lateral sclerosis. PLoS ONE 19(6): e0304593. https://doi.org/10.1371/journal.pone.0304593
Editor: Farzin Hajebrahimi, New Jersey Institute of Technology, UNITED STATES
Received: January 22, 2024; Accepted: May 14, 2024; Published: June 20, 2024
Copyright: © 2024 Serian et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The data underlying the results presented in the study are available from https://doi.org/10.18725/OPARU-52545.
Funding: This is an EU Joint Programme - Neurodegenerative Disease Research (JPND) project (“NEEDSinALS” 01ED1405). The project is supported through the following organizations under the aegis of JPND - www.jpnd.eu e.g. Germany, Bundesministerium für Bildung und Forschung (BMBF, FKZ), Sweden, Vetenskaprådet Sverige, Poland, Narodowe Centrum Badan´ i Rozwoju (NCBR). This work was additionally funded by the Deutsche Forschungsgemeinschaft (DFG, LU 336/13-2), the Bundesministerium für Bildung und Forschung (PaCEMed, 01DS18031; FTLDc O1GI1007A, MND-Net 01GM1103A). The funding sources played no role in the preparation of this manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
It is now recognized that Amyotrophic Lateral Sclerosis (ALS) is a multisystemic disorder with neuropsychological dysfunction as the most common comorbidity in 35% to 45% of ALS patients [1]. These are being classified as cognitively impaired (ALSci; Amyotrophic Lateral Sclerosis with cognitive impairment) without fulfilling the criteria for dementia [1], while 5% up to 14% suffer from dementia, mostly in the sense of behavioural variant frontotemporal dementia (bvFTD) [1–3]. For ALSci, the most frequently reported cognitive impairments are deficits of the executive functions including verbal fluency and working memory as well as deficits of language and social cognition [4–6], depending on disease stage [7]. In bvFTD, behavioural symptoms such as personality change, irritability and pervasive deficits are the primary diagnostic criteria, although other cognitive impairments, especially in executive functions, may be present too [8]. These neuropsychological impairments may interfere with daily routine when they reach a certain threshold. Thus, for clinical practice it is essential to identify cognitive impairment and its severity.
To determine the cognitive status especially in ALS patients, the Edinburgh Cognitive and Behavioural ALS Screen (ECAS) was developed [9] and is now a well-established instrument with adaptations and translations for more than 20 languages [10]. It is designed to detect both ALS specific cognitive impairment, found in language, verbal fluency and executive functions, as well as ALS non-specific impairment in memory and visuospatial functions [9]. A major advantage of the ECAS is that it takes hand and speech motor impairment into account, allowing cognitive screening in severely physically impaired patients [9]. The most frequent form of dementia in ALS, bvFTD, can be determined with carer behaviour scale of the ECAS.
Although the ECAS was specially designed for ALS patients, in clinical practice and also in research with ALS patients, cognitive screening is often done via Mini-Mental State Examination (MMSE) which is one of the best known screening instruments validated for patients with neurological and psychiatric disorders [11, 12]. In our meta-analysis about cognition in ALS patients from 2022, we found that 31% of included studies used the MMSE for a cognitive screening or outcome measure, while only 20% of studies used the ECAS [6]. Reasons for the high application rate of the MMSE, even to screen ALS patients, may be its fast and easy application, its high acceptance among health professionals and researchers, its availability in multiple languages, and its use for the assessment of the severity of dementia [12–14]. However, the MMSE is motor dependent, so patients with progressive motor disability are unable to perform it in the course of their disease. Furthermore, the MMSE may be insensitive for clinical characteristics of frontotemporal dementia, like executive dysfunction or social cognition impairment in bvFTD [15], but some studies suggests that, nevertheless, it seems to be an appropriate measure to monitor severity and disease progression in bvFTD [16, 17].
The goal of this study was to compare the screening performance of the widely used MMSE and the specifically designed ECAS for cognitive dysfunction in ALS patients, to understand the relationship of both instruments, and to investigate the extent to which the test values of the ECAS correspond to the test values of the MMSE for cognitive impairment of ALS patients.
Methods
Participants
In total, 83 patients were recruited in the Department of Neurology at the University Hospital of Ulm between 21st February 2018 and the 27th May 2021. Interviews and screenings were performed by (neuro-)psychologists, health professionals and clinical neurologists in training. Total interview and screening lasted 30–60 minutes, depending on the physical impairments of the subjects. The study was approved by the Ethics Committees of the Ulm University (19/12), in accordance with the ethical standards of the revised Helsinki Declaration in 1983. All participants gave written informed consent prior to inclusion in the study. Inclusion criteria for the study were German as a first or comparably good second language, and no history of any psychiatric or neurological disorders, apart from ALS and (reactive) depression. We had to exclude one patient due to foreign first language and one patient due to rejected ALS-diagnosis. None of the patients had a clinical diagnosis of dementia at time of study inclusion, however two patients were afterwards diagnosed with dementia, one with FTD and one with primary progressive aphasia (PPA). In order to improve representation of patients with severe cognitive impairments up to dementia in the study, another three patients with ALS and suspected dementia were examined from 7th March 2024 to 20th March 2024. One of these patients received a clinical diagnosis of behavioural variant of FTD (bvFTD), for the other two patients, final findings regarding dementia development were still pending. In summary, 84 patients with the diagnosis of ALS according to the revised El Escorial criteria [18] could be included.
Demographics concluded of age, education, sex, months since symptom onset, site of onset, family history of ALS, severity of physical impairment (measured by the revised ALS Functional Rating Scale (ALSFRS-R, with a range from 48 (no impairment) to 0 (extreme impairment)), progression rate (calculated as 48 minus current ALSFRS-R score divided by months since symptom onset), usage of non-invasive ventilation (NIV) and/or percutaneous endoscopic gastrostomy (PEG), and depressiveness (measured by ALS-Depression-Inventory (ADI-12, with a range from 12 (no depression) to 48 (severe depression)) [19].
Edinburgh Cognitive and Behavioural ALS Screen (ECAS)
The ECAS is a 136-point test consisting of 16 tasks covering the cognitive domains of language, verbal fluency and executive functions, subsumed under the ALS specific score, as well as memory, subsumed und the ALS non-specific score [9]. In this study, the validated German version of the ECAS was used [3] with established age- and education-adjusted cut-offs for all subscores and the total ECAS score to define cognitive impairment [20]. The ECAS can be performed either in a written or spoken version to minimize the influence of motor impairment. In this study, all participants performed the spoken version.
Mini-Mental State Examination (MMSE)
The Mini Mental State Examination (MMSE) is a 30-point test consisting of 19 individual tasks covering orientation, calculation, memory, visuoconstruction and basic language and action tasks [11]. In this study the German version of MMSE was used [21]. Based on Jia et al. [22], the tasks were assigned to five domains, the tasks were subsumed under cognitive domains, namely orientation (orientation to place and time, 10 points), memory (repeat and recall 3 words, 6 points), visuospatial skills (copy intersecting pentagons, 1 point), language (naming, repetition, write a sentence, 4 points) and executive functions (serial 7 substraction, 3-step command test, reading command test, 9 points). Impairment of a cognitive domain was defined as -1 standard deviation of the mean. According to Kilada et al. [23], an overall MMSE score of less than 27 is considered a validated diagnostic standard for cognitive impairment. If patients were not able to complete a task of the MMSE due to physical decline and motor restrictions, they were grouped into a separate group “patients without motor capacity for MMSE” and their results of the MMSE were not used in the final analysis.
Statistics
Data distribution of metric demographic and cognitive variables were tested with the Shapiro-Wilk test. Group differences between patients with motor capacity to complete the MMSE and patients without the motor capacity to complete the MMSE were tested with Chi-Square test for nominal data, with Mann-Whitney-U test for non-normal distributed data, and with two-tailed t-test for normal-distributed data. A threshold of α = 0.05 (two-tailed) was applied for statistical interference. Cohen’s d was used as effect size parameter in significant statistical tests with 0.2, 0.5 and 0.8 as thresholds for a small, medium, and large effect, respectively. Curve estimation regression (invers model, s-curve model, and cubic model) was performed to describe the relationship between MMSE and ECAS, with ECAS as predictor, choosing the best curve fit by the highest R2. The data was analyzed using R 3.6.3 and IBM SPSS statistics version 25.
Results
Demographics
Mean values and standard deviations (SD) of demographics of included patients are shown in Table 1. Thirty-six patients (42.9%) in this sample showed depressive symptoms according to a cut-off score ≥ 23 for a valid hint for depressive symptoms [24], while twelve patients (14.3%) showed clinical relevant depressive symptoms with a cut-off score >29 [24] assessed by ADI-12.
Nine participants (10.7%) were not able to complete the MMSE due to loss of needed motor capacity, their demographics are shown separately in Table 1. These patients had a longer disease duration (months of onset: U = 92.50, p = .006, d = 0.84) and a lower ALSFRS-R (U = 491.00, p = .03, d = 0.50), but did not differ in terms of age (t(82) = -0.70, p = .50), sex (χ2(1) = 0.74, p = .39), education (U = 386.00, p = .49), progression rate (U = 285.00, p = .62), family history of ALS (χ2(1) = 1.38, p = .24), NIV and PEG use (χ2(1) = 0.00, p > .99 and χ2(1) = 0.44, p = .51, respectively), depressiveness (U = 276.50, p = .38) and ECAS total score (U = 369.50, p = .65).
Performance on ECAS and MMSE
The mean ECAS total score was 98.53 (SD = 20.43, range 28–128), 17 participants (22.7%) were under the respective German age- und education-adjusted cut-offs [3] for impaired cognition (Table 2). The mean MMSE total score in this sample was 27.68 (SD = 2.57, range 17–30). Seventeen participants (22.7%) showed a MMSE total score under the cut-off score of < 27 for cognitive impairment [23] (Table 2). A total of 9/75 (12.0%) participants showed impairment in both tests, while another 8/75 (10.7%) who tested normal in the MMSE were impaired in the ECAS, and 58/75 (77.3%) tested normal in both MMSE and ECAS (Table 3).
As seen in Table 2, between 17.3% (executive functions) to 36.0% (verbal fluency) of patients were impaired in the individual domains of the ECAS, while between 11.1% (visuospatial skills and executive functions) and 27.8% (memory) were impaired in individual domains of the MMSE. Dysfunction in verbal fluency, which was only measured by the ECAS, was found in 36.0% of patients, while problems with orientation, measured only by the MMSE, were found in 12.5% of patients. The three patients who were diagnosed with dementia after completing the study were shown to be far below the respective cut-off scores for cognitive impairment in both screening instruments (FTD-patient: ECAS total score: 65, MMSE total score: 22; bvFTD-patient: ECAS total score: 48, MMSE total score: 23; PPA-patient: ECAS total score: 28, MMSE total score: 17). One of the two patients with pending findings regarding dementia was below the respective cut-offs in both instruments (ECAS total score: 74, MMSE total score: 25), however the other patient only showed cognitive impairment using the ECAS (ECAS total score: 69, MMSE total score: 29).
The relationship of the MMSE total score and the ECAS total score was shown to be non-linear (Fig 1). The MMSE total score was predicted by the ECAS total score (1/ECAS total score: β = -.83, t = -12.51, p < .001) by using an s-curve regression model (F (1,73) = 156.41, p < .001) with an adjusted R2 = .68 (plotted in Fig 1). The corresponding values of ECAS and MMSE calculated by this model are shown in Table 4, with the ECAS being more discriminative in the higher range of cognitive performance (range 80–136 ECAS total score), for which the MMSE does not define any cognitive impairment (corresponding to MMSE score 27–30).
S-curve regression (F(1,73) = 156.41, p < .001, R2 = .68); ECAS = Edinburgh Cognitive and Behavioural ALS Screen. MMSE = Mini-Mental State Examination.
Discussion
The purpose of this study was to compare the screening performance of the MMSE and the ECAS in ALS patients and to understand the relationship of both instruments. While the ECAS could be administered to the entire cohort, 89.3% of the patients were able to complete the MMSE in its original form. ALS patients who were not able to complete the MMSE had, as expected, a longer disease duration and were physically more impaired, however did not differ in general demographics, depressiveness, or cognition measured by the ECAS.
This study found a non-linear relationship between the performance on the ECAS and the MMSE of ALS patients with a relatively high R2 of .68. The cognitive domains of memory, language, executive functioning and visuospatial functioning, which all were included in both tests to a certain degree, showed relatively similar rates of impairment with 5 to 7.5% difference. Also, both MMSE and ECAS were able to identify all three patients with a diagnosed frontotemporal dementia variant with substantially decreased total scores, which could support the use of the MMSE in the detection or at least the monitoring of FTD in patients [16, 17]. If we assume a cut-off of 22 or 23 points [25, 26] for dementia in the MMSE, which fits our results, values of 46–55 in the total ECAS score may be indicative for dementia according to our calculated corresponding values. This, however, must be examined in a larger study regarding age and education.
While the rate of cognitively impaired patients was the same for both tests at approximately 23%, the agreement as to which patient is impaired and which is normal was relatively low. Only 53% of patients who showed cognitive impairment according to age- and education-based cut-offs of the ECAS in the total score were also impaired in the MMSE with an applicated cut-off of less than 27 points. Additionally, 47% of patients impaired in the ECAS total score were considered “normal” in the MMSE. An explanation could be the difference in tested cognitive domains in both tests. While orientation to time and place makes up a third of points in the MMSE, it is of no relevance in the ECAS. On the other hand, verbal fluency, a cognitive domain most typically found impaired not only in ALS patients [4–6, 27] but also in presymptomatic C9orf72 gene carriers [27, 28], and which was substantially decreased in 36% of patients in this study, cannot be examined by the MMSE. Also, the MMSE is known to suffer from ceiling effects [12, 25, 29, 30], especially in patients without full developed dementia [25, 29], which furthermore could explain the discrepancy in detection of cognitive impairment. In addition, meta-analyses show satisfactory values for sensitivity (77% [25] / 82% [26] / 85% [26]) and specificity (91% [25] / 90% [26] / 82% [26]) regarding the identification of dementia by the MMSE, but much lower values regarding the detection of mild cognitive impairment (sensitivity: 63%, specificity: 65%) [25]. Thus, application of the MMSE as a screening instrument for cognitive impairment in ALS patients is not ideal, and the usage of the ECAS should be preferred. If a shorter diagnostic tool must be used, the Montreal Cognitive Assessment (MoCA) was shown to be an acceptable alternative screening instrument for cognitive impairment in non-demented ALS patients, but also poses the problem of motor-dependence [30]. However, our results do not oppose the application of the MMSE as instrument for ruling out or monitoring dementia in ALS patients, which should be examined in further studies.
Limitations
This cohort was tested in rather early stages of the disease with a still rather high motor capacity. Although the feasibility of the MMSE is expected to decline with further progression, present findings should not be extrapolated to late-stage patients. Although the rate of cognitive impairment and dementia matches the expected prevalence in ALS patients in general [1–3], the number of patients with cognitive impairments and dementia is low due to the overall relatively small number of patients. Further investigation of the relationship between the performance in MMSE and ECAS of ALS patients in a larger population, preferably with previously clearly defined subgroups with dementia, is recommended to potentially strengthen the current results.
Conclusion
In summary, even though the MMSE is one of the most known and applied screening instruments, its application as a screener for cognitive impairment in early-stage ALS patients is not ideal, as it does not cover the full range of cognitive changes that may present in ALS patients and is also motor-dependent. The preferred tool should be the specially designed ECAS. However, if clinicians or researchers have no expertise in applying the ECAS or interpreting ECAS results, or if only the MMSE was used, it can be helpful to relate the ECAS score to the MMSE value or vice versa. The application of the MMSE as a screener to rule out dementia in ALS patients should be examined in further studies, also the creation of a cut-off for dementia in the ECAS should be considered, so that research studies and drug trials do not have to rely on screening instruments that are not suitable for ALS patients.
Acknowledgments
We would like to thank all patients and carers who supported us with their participation in the study. We would also like to thank Caritt Reichl for assistance in collecting data.
References
- 1. Strong MJ, Abrahams S, Goldstein LH, Woolley S, Mclaughlin P, Snowden J, et al. Amyotrophic lateral sclerosis—frontotemporal spectrum disorder (ALS-FTSD): Revised diagnostic criteria. Amyotrophic Lateral Sclerosis and Frontotemporal Degeneration. 2017 Apr 3;18(3–4):153–74. pmid:28054827
- 2. Phukan J, Elamin M, Bede P, Jordan N, Gallagher L, Byrne S, et al. The syndrome of cognitive impairment in amyotrophic lateral sclerosis: a population-based study. J Neurol Neurosurg Psychiatry. 2012 Jan 1;83(1):102–8. pmid:21836033
- 3. Lulé D, Burkhardt C, Abdulla S, Böhm S, Kollewe K, Uttner I, et al. The Edinburgh Cognitive and Behavioural Amyotrophic Lateral Sclerosis Screen: A cross-sectional comparison of established screening tools in a German-Swiss population. Amyotrophic Lateral Sclerosis and Frontotemporal Degeneration. 2015 Mar 31;16(1–2):16–23. pmid:25292386
- 4. Goldstein LH, Abrahams S. Changes in cognition and behaviour in amyotrophic lateral sclerosis: nature of impairment and implications for assessment. The Lancet Neurology. 2013 Apr 1;12(4):368–80. pmid:23518330
- 5. Beeldman E, Raaphorst J, Twennaar MK, Visser M de, Schmand BA, Haan RJ de. The cognitive profile of ALS: a systematic review and meta-analysis update. J Neurol Neurosurg Psychiatry. 2016;87(6):611–9. pmid:26283685
- 6. Finsel J, Uttner I, Vázquez Medrano CR, Ludolph AC, Lulé D. Cognition in the course of ALS—a meta-analysis. Amyotrophic Lateral Sclerosis and Frontotemporal Degeneration. 2022 Jul 22;0(0):1–12. pmid:35866707
- 7. Lulé D, Böhm S, Müller HP, Aho-Özhan H, Keller J, Gorges M, et al. Cognitive phenotypes of sequential staging in amyotrophic lateral sclerosis. Cortex. 2018 Apr 1;101:163–71. pmid:29477906
- 8. Mioshi E, Hsieh S, Savage S, Hornberger M, Hodges JR. Clinical staging and disease progression in frontotemporal dementia. Neurology. 2010 May 18;74(20):1591–7. pmid:20479357
- 9. Abrahams S, Newton J, Niven E, Foley J, Bak TH. Screening for cognition and behaviour changes in ALS. Amyotrophic Lateral Sclerosis and Frontotemporal Degeneration. 2014 Mar 1;15(1–2):9–14. pmid:23781974
- 10. ECAS International–ECAS [Internet]. [cited 2023 Jul 20]. Available from: https://ecas.psy.ed.ac.uk/ecas-international/
- 11. Folstein MF, Folstein SE, McHugh PR. “Mini-mental state”: a practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research. 1975 Nov;12(3):189–98.
- 12. Nieuwenhuis-Mark RE. The Death Knoll for the MMSE: Has It Outlived Its Purpose? J Geriatr Psychiatry Neurol. 2010 Sep 1;23(3):151–7. pmid:20231732
- 13. Arevalo‐Rodriguez I, Smailagic N, Roqué i Figuls M, Ciapponi A, Sanchez‐Perez E, Giannakou A, et al. Mini‐Mental State Examination (MMSE) for the detection of Alzheimer’s disease and other dementias in people with mild cognitive impairment (MCI). Cochrane Database Syst Rev. 2015 Mar 5;2015(3):CD010783. pmid:25740785
- 14. Perneczky R, Wagenpfeil S, Komossa K, Grimmer T, Diehl J, Kurz A. Mapping Scores Onto Stages: Mini-Mental State Examination and Clinical Dementia Rating. The American Journal of Geriatric Psychiatry. 2006 Feb 1;14(2):139–44. pmid:16473978
- 15. Tan KS, Libon DJ, Rascovsky K, Grossman M, Xie SX. Differential Longitudinal Decline on the Mini-Mental State Examination in Frontotemporal Lobar Degeneration and Alzheimer’s Disease. Alzheimer Dis Assoc Disord. 2013;27(4): pmid:23314064
- 16. Osher JE, Wicklund AH, Rademaker A, Johnson N, Weintraub S. The Mini-Mental State Examination in Behavioral Variant Frontotemporal Dementia and Primary Progressive Aphasia. Am J Alzheimers Dis Other Demen. 2008 Dec 1;22(6):468–73.
- 17. Chow TW, Hynan LS, Lipton AM. MMSE Scores Decline at a Greater Rate in Frontotemporal Degeneration Than in AD. Dement Geriatr Cogn Disord. 2006;22(3):194–9. pmid:16899996
- 18. Ludolph A, Drory V, Hardiman O, Nakano I, Ravits J, Robberecht W, et al. A revision of the El Escorial criteria—2015. Amyotrophic Lateral Sclerosis and Frontotemporal Degeneration. 2015 Aug 27;16(5–6):291–2. pmid:26121170
- 19. Kübler A, Winter S, Kaiser J, Birbaumer N, Hautzinger M. Das ALS-Depressionsinventar (ADI): Zeitschrift für Klinische Psychologie und Psychotherapie. 2005 Jan 1;34(1):19–26.
- 20. Loose M, Burkhardt C, Aho-Özhan H, Keller J, Abdulla S, Böhm S, et al. Age and education-matched cut-off scores for the revised German/Swiss-German version of ECAS. Amyotroph Lateral Scler Frontotemporal Degener. 2016 Aug;17(5–6):374–6. pmid:27027323
- 21.
Kessler J, Markowitsch HJ, Denzler P. MMST—Mini-Mental-Status-Test: Deutschsprachige Fassung [Internet]. Weinheim: Beltz; 1990 [cited 2021 May 18]. Available from: https://www.testzentrale.de/shop/mini-mental-status-test.html
- 22. Jia X, Wang Z, Huang F, Su C, Du W, Jiang H, et al. A comparison of the Mini-Mental State Examination (MMSE) with the Montreal Cognitive Assessment (MoCA) for mild cognitive impairment screening in Chinese middle-aged and older population: a cross-sectional study. BMC Psychiatry. 2021 Oct 4;21(1):485. pmid:34607584
- 23. Kilada S, Gamaldo A, Grant EA, Moghekar A, Morris JC, O’Brien RJ. Brief Screening Tests for the Diagnosis of Dementia: Comparison With the Mini-Mental State Exam. Alzheimer Disease & Associated Disorders. 2005 Mar;19(1):8–16. pmid:15764865
- 24. Hammer EM, Häcker S, Hautzinger M, Meyer TD, Kübler A. Validity of the ALS-Depression-Inventory (ADI-12)—A new screening instrument for depressive disorders in patients with amyotrophic lateral sclerosis. Journal of Affective Disorders. 2008 Jul 1;109(1):213–9. pmid:18262283
- 25. Mitchell AJ. A meta-analysis of the accuracy of the mini-mental state examination in the detection of dementia and mild cognitive impairment. Journal of Psychiatric Research. 2009 Jan 1;43(4):411–31. pmid:18579155
- 26. Trivedi D. Cochrane Review Summary: Mini-Mental State Examination (MMSE) for the detection of dementia in clinically unevaluated people aged 65 and over in community and primary care populations. Prim Health Care Res Dev. 2017 Nov;18(06):527–8. pmid:28578720
- 27. De Marchi F, Carrarini C, De Martino A, Diamanti L, Fasano A, Lupica A, et al. Cognitive dysfunction in amyotrophic lateral sclerosis: can we predict it? Neurol Sci. 2021;42(6):2211–22. pmid:33772353
- 28. Lulé DE, Müller HP, Finsel J, Weydt P, Knehr A, Winroth I, et al. Deficits in verbal fluency in presymptomatic C9orf72 mutation gene carriers—a developmental disorder. J Neurol Neurosurg Psychiatry. 2020 Nov 1;91(11):1195–200. pmid:32855285
- 29. Diniz BSO, Yassuda MS, Nunes PV, Radanovic M, Forlenza OV. Mini-mental State Examination performance in mild cognitive impairment subtypes. Int Psychogeriatr. 2007 Aug;19(4):647–56. pmid:17502007
- 30. Aiello EN, Solca F, Torre S, Carelli L, Ferrucci R, Priori A, et al. Diagnostics and clinical usability of the Montreal Cognitive Assessment (MoCA) in amyotrophic lateral sclerosis. Front Psychol. 2022 Sep 23;13:1012632. pmid:36211901