https://orcid.org/0000-0001-8599-6107
Figures
Abstract
This study examined the ability of a computer-based cognitive assessment tool (CompBased-CAT) to predict mild cognitive impairment (MCI) in community-dwelling older adults. A two-year longitudinal study was conducted using data from 2016 to 2018 from the Otassha study cohort of community-dwelling older adults. MCI was defined as a Mini-Mental Status Examination (MMSE) score of <27. The CompBased-CAT was used at baseline, with each subtest score converted into a Z-score. Subsequently, the total Z-scores were calculated. Participants were divided into robust and MCI groups, and all variables were compared using the t-test or χ2 test. Receiver operating characteristic (ROC) curves and logistic regression analyses were conducted, with MCI and total Z-scores as dependent and independent variables, respectively. Among the 455 participants (median age, 72 years; range, 65–89 years; 282 women and 173 men), 32 developed MCI after two years. The participants in the MCI group were significantly older. They had lower maximal gait speed, baseline MMSE scores, subtest Z-scores, and total Z-scores than those in the robust group. The area under the ROC curve was 0.79 (95% confidence interval: 0.70–0.87; P <0.01). The sensitivity was 0.76, and the specificity was 0.75. The logistic regression analysis showed an odds ratio of 1.34 (95% confidence interval: 1.18–1.52; P <0.01). This study showed that CompBased-CAT can detect MCI, which is an early stage of dementia. Thus, CompBased-CAT can be used in future community health checkups and events for older adults.
Citation: Takahashi J, Kawai H, Ejiri M, Fujiwara Y, Hirano H, Sasai H, et al. (2024) Predicting the incidence of mild cognitive impairment with a computer-based cognitive assessment tool in community-dwelling older adults: The Otassha study. PLoS ONE 19(1): e0297433. https://doi.org/10.1371/journal.pone.0297433
Editor: Valerio Manippa, University of Bari Department of Education Psychology and Communications: Universita degli Studi di Bari Aldo Moro Dipartimento di Scienze della Formazione Psicologia Comunicazione, ITALY
Received: August 31, 2023; Accepted: January 4, 2024; Published: January 25, 2024
Copyright: © 2024 Takahashi et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The data from this cohort study in Itabashi Ward contain sensitive participant information and cannot be released publicly due to ethical and legal restrictions imposed by the Ethics Committee of the Tokyo Metropolitan Institute of Gerontology. All the relevant data are available upon reasonable request to the Health Promotion Management Office (kaigoyob@tmig.or.jp), the department responsible for this cohort study at the Tokyo Metropolitan Institute of Gerontology.
Funding: This work was supported by Japan Society for the Promotion of Science KAKENHI (award-winner: SO, grant numbers: 15K01469 and 16K01853, URL: https://www.jsps.go.jp/english/) and Research Funding for Longevity Sciences (award-winner: SO, grant numbers: 28-30 and 29-42, URL: https://www.ncgg.go.jp/english/index.html), from the National Center for Geriatrics and Gerontology, Japan. The funders had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
As the global population continues to age, the number of people with dementia has been increasing, with approximately 1.52 billion people estimated to have dementia by 2050 [1]. The decline in the quality of life (QOL) of patients with dementia, the burden of care assumed by their families, and socioeconomic burden have emerged as major concerns [2, 3], making dementia prevention an urgent issue.
Approximately 10–15% of individuals with mild cognitive impairment (MCI) progress to Alzheimer’s disease annually [4]; therefore, early diagnosis and management of MCI is important in preventing dementia. Approximately 35% of risk factors for dementia are modifiable [5], and interventions such as a Mediterranean diet [6], exercise [7], and social participation [8] have been reported to reduce the risk of dementia. Therefore, the early detection of MCI can be expected to reduce the risk of developing dementia or delay its progression with appropriate interventions and social support. Earlier interventions provided people with dementia and their families with more time to prepare for the future [5]. However, dementia is often not diagnosed until its late stages, rendering the diagnosis considerably less useful [9]. Delays of several years in screening and diagnosis can reduce the effectiveness of interventions. Therefore, cognitive impairments should be detected as early as possible. Currently, paper-based tests such as the Mini-Mental State Examination (MMSE) [10] and Montreal Cognitive Assessment (MoCA) [11] are widely used to screen for MCI. However, paper-based assessment tools have disadvantages such as learning effectiveness, measurement bias, and the requirement of trained staff [12], making them unsuitable for frequently testing older adults or validating the effectiveness of interventions.
Therefore, we developed a computer-based cognitive assessment tool (CompBased-CAT), a cognitive function testing tool that uses a tablet computer [13]. The tool does not require a trained examiner and has no measurement bias because the examinee is guided and assessed using a computer. In addition, the tasks were selected randomly; therefore, the learning effect was considered. Our previous study [13] revealed that this tool moderately correlated with the MMSE score (r = 0.51) and had sufficient discriminating ability for dementia (sensitivity = 0.81, specificity = 0.77). However, the previous study was cross-sectional, and whether the CompBased-CAT can predict future MCI remains unclear. Although previous studies have used tablet computers to identify the occurrence of dementia with advanced cognitive impairment, no study has examined MCI. Detecting risk at the MCI stage is potentially useful for dementia prevention.
This study aimed to examine the ability of the CompBased-CAT to predict the future occurrence of MCI in community-dwelling older adults. Moreover, the association between CompBased-CAT scores at baseline and the occurrence of MCI two years later was examined. We hypothesize that CompBased-CAT has a practical predictive ability for future MCI incidence.
Materials and methods
Study design and participants
We conducted a two-year longitudinal study. Participants were selected from the 2016 survey of the Otassha Study [14]. This cohort study began in 2011 and is conducted annually in September and October at the Tokyo Metropolitan Institute of Gerontology (Tokyo, Japan). Participants were recruited by sending invitations to community-dwelling older adults aged ≥65 years living in nine areas of Itabashi Ward, Tokyo. Invitations were sent annually to those who had previously participated in this study and those who were newly 65 (except in 2015). Participants were recruited for the 2016 survey between October 19 and November 10, 2016. The inclusion criteria were participation in the 2016 and 2018 surveys and the absence of MCI at baseline (MMSE score ≥27). In contrast, those with a history of dementia and those with missing data were excluded. The presence of dementia was verified through an interview with a nurse, checking for medical history and medications. The participants were informed of the study in advance, and their written informed consent was subsequently obtained. We did not have access to information that could identify the individual participants during or after data collection. The study protocol was approved by the Ethics Committee of the Tokyo Metropolitan Institute of Gerontology (approval number 2015–18).
CompBased-CAT measurement
The CompBased-CAT is a multidomain cognitive function test that uses a tablet computer (ASUS TransBook T100HA China, OS: Windows 10) and consists of six subtests: digit span forward (attention and concentration), digit span backward (attention and concentration), memory of item names (immediate memory), memory recall of item names (remote memory), Stroop (executive function and selective attention), and recognition of figures (space perception) [13]. The participants wore noise-canceling headphones and responded to the tasks while receiving audio guidance. They first performed practice tasks and subsequently performed the main task. The CompBased-CAT was conducted at an inspection venue during the 2016 survey. The CompBased-CAT scores were converted to Z-scores for each subtest based on data from the overall study population, and the total Z-score was calculated by adding the Z-scores of all subtests without any weighting [13].
Measuring the occurrence of cognitive impairment
Cognitive function was evaluated using the MMSE. MCI was defined as an MMSE score of <27. The discriminatory ability of the MMSE for MCI when using this cut-off value has been reported previously, with sensitivities and specificities of 69.1–85.7% and 77.3–80.1%, respectively [15]. The MMSE was administered through face-to-face interviews with trained staff at inspection venues in 2016 and 2018.
Other measurements
Covariates including age, sex, number of chronic diseases (cerebral stroke, cardiac diseases, diabetes mellitus, and depression), years of education, maximal gait speed, simple Japanese version of the World Health Organization (WHO)-Five well-being index (WHO-5) [16], and Tokyo Metropolitan Institute of Gerontology Index of Competence (TMIG-IC) [17] at the time of the 2016 survey were assessed. The WHO-5 is a short version of the WHO Well-Being Scale that was initially developed to evaluate the quality of care for diabetes and has been validated in the context of various health states, including depression, anxiety, psychiatric disorders, and health-related QOL. Scores range from 0 to 15, with higher scores indicating better mental health [16]. The TMIG-IC was designed based on Lawton’s model of human behavior and was used to assess the ability of older adults to perform instrumental activities of daily living (IADL). The scores range from 0 to 13, with higher scores indicating better IADL ability [17]. Maximum gait speed was calculated by measuring the time required to walk a 5 m path with maximum effort. Preliminary 3 m paths for acceleration and deceleration were provided at the beginning and end. Walking speed was measured twice, and the maximum value was adopted. The other items were acquired through face-to-face interviews with the researcher.
Statistical analysis
Participants were divided into robust and MCI groups, and all variables were compared between the groups using the t-test or χ2 test. Receiver operating characteristic (ROC) curve analysis was performed to predict the occurrence of MCI after two years based on the total Z-score of the CompBased-CAT, and the area under the curve (AUC) was calculated. The cut-off point, sensitivity, and specificity were calculated using the Youden index. As a sub-analysis, logistic regression analysis was performed to determine the association between CompBased-CAT scores (total Z-scores and subtest scores) and the occurrence of MCI. In model 1, age, sex, and years of education were included as covariates. In model 2, the number of chronic diseases (cerebral stroke, cardiac diseases, diabetes mellitus, and depression), grip strength, maximum gait speed, WHO-5 score, and TMIG-IC score were included as additional covariates. Statistical significance was set at P <0.05. All analyses were performed using the IBM SPSS software (version 25.0; IBM Japan, Tokyo, Japan).
Results
Participant characteristics
In the baseline survey, the total number of participants were 831, of whom 187 were excluded (missing data: 73; MMSE score: <27:114). Of the 644 eligible participants at baseline, 184 (28.5%) were lost to follow-up. Finally, 455 participants were included in the analysis (median age: 72 years; range: 65–89 years; 282 women and 173 men), 32 of whom had new MCI (Fig 1, Table 1). The mean years of education for all participants were 13.4 ± 2.8 years. The mean MMSE was 29.0 ± 1.0, and the mean score for TMIG-IC was 12.5 ± 1.0. Participants in the MCI group were significantly older and had lower maximum gait speed, baseline MMSE scores, subtest Z-scores, and total Z-scores than those in the control group. Table 2 shows the CompBased-CAT scores at baseline. The total Z-score ranged from −13.7 to 7.42, and the mean total Z-score was 0.27 ± 3.26. The Z-scores in all subtests and the total Z-score from the CompBased-CAT were significantly lower in the MCI group than in the robust group.
Predictability of the CompBased-CAT
Fig 2 shows the ROC curves for predicting the occurrence of MCI and the performance indicators of CompBased-CAT. The AUC was 0.79 (95%CI: 0.70–0.87, P <0.01), and the cut-off total Z score was -1.67, with sensitivity and specificity of 0.76 and 0.75, respectively.
AUC, area under the receiver operating characteristic curve; CI, confidence interval.
Association between CompBased-CAT scores and the risk of MCI
The total and subtest Z-scores of the CompBased-CAT were significantly associated with MCI risk in the crude model and model 1 (Table 3). In model 2, significant associations were observed for the total Z-score, digit span forward, digit span backward, memory recall of item names, recognition of figures, and Stroop task. However, no significant associations were observed for the memory of item names. In comparisons by subtest, odds ratios were higher for the digit span backward, digit span forward, memory recall of an item name, recognition of figures, and the Stroop task.
Discussion
Principal findings
In this study, we examined the predictive ability of the CompBased-CAT and the relationship between CompBased-CAT subtests and the risk of MCI after two years in community-dwelling older adults. CompBased-CAT accurately predicted MCI occurrence within two years. Although several PC-based cognitive function tests have been reported, most studies were cross-sectional, and participants attended memory clinics [18]. This study demonstrated that PC-based cognitive function tests can be used to assess future MCI risk in community-dwelling older adults. We believe that by predicting the risk at the MCI stage, proactive measures against dementia can be implemented effectively.
Participant characteristics
The participants had mean years of education and MMSE and TMIG scores of 13.4 years, 29.0, and 12.5, respectively, which were slightly better than those reported in similar studies [19, 20]. This may be because this study was an invitational venue survey, which may have been more likely to include participants who were highly interested in health and could commute to the venue. The incidence of MCI in this study was 35.2 per 1000 person-years. A systematic review reported that the incidence of MCI in population- or community-based studies was between 8.5 and 76.8 per 1000 person-years [21]. The incidence of MCI in this study was within this range. Although the participants in this study had slightly better physical and psychosomatic functions, the incidence of MCI showed a general value, indicating that the study participants comprised a reasonable sample.
Predictive ability of the CompBased-CAT
The CompBased-CAT had an AUC, sensitivity, and specificity of 0.79, 0.76, and 0.75, respectively. Previous studies that examined the ability of PC-based cognitive function tests to discriminate MCI reported higher accuracies, with AUCs, sensitivities, and specificities of 0.80–0.97, 0.80–0.96, and 0.73–0.94, respectively [18]. However, these results were obtained from cross-sectional studies and cannot be meaningfully compared with the results of this study. Shimada et al. [22] longitudinally examined the predictive ability of the National Center for Geriatrics and Gerontology-Functional Assessment Tool (NCGG-FAT), a PC-based cognitive function test, to predict the development of dementia and reported an AUC of 0.57–0.71. This study similarly examined the predictive ability of CompBased-CAT, a PC-based cognitive function test, for the future development of MCI and exhibited better predictive ability (AUC = 0.79) than in NCGG-FAT. This tool is significant because it targets MCI and is considered useful for preventing dementia in older people. Recently, a machine learning-based approach was used to predict cognitive impairment using multiple variables, with high predictive accuracy of AUC (0.92), sensitivity (0.97), and specificity (0.83) [23]. This approach is based on the prediction of 35 covariates and is not useful for screening risks in clinical practice because it requires multiple assessments. CompBased-CAT can be performed in 10–15 min and does not require specialized staff, making it suitable for actual clinical settings.
Comparison in each subtest of the CompBased-CAT
The baseline scores of each subtest of the CompBased-CAT were significantly lower in the MCI group, indicating that those at risk for MCI show functional impairment in multiple domains of cognitive function from an early stage. Previous studies [24, 25] have reported that MCI in multiple domains, such as memory, verbal fluency, and figure recognition, was observable several years before the diagnosis of dementia, and similar results were obtained using CompBased-CAT. As early symptoms of cognitive impairment are observed in multiple domains of cognitive function, an accurate assessment of MCI across many domains may not be possible by testing only a small number of selected domains. This tool assesses cognitive function in multiple domains and is useful for detecting the early stages of MCI.
The logistic regression analysis showed that the total Z-score was significantly associated with the occurrence of MCI, even after controlling for all covariates, and that the total Z-score could predict the occurrence of MCI independent of demographic factors, physical and mental functioning, and IADL ability. In addition, for each subtest Z-score of the CompBased-CAT, a significant association was observed with the occurrence of MCI, even after adjusting for all covariates for digit span forward, digit span backward, memory recall of item names, recognition of figures, and the Stroop task. The association with MCI was stronger for the forward (odds ratio [OR] = 2.11, P <0.01) and backward digit spans (OR = 2.21, P <0.01). Because digit span forward and backward tasks require memorizing numbers and processing information simultaneously (i.e., working memory) [26], they may predict MCI more accurately than simple memory skills such as immediate memory. Working memory is impaired in the early stages of Alzheimer’s dementia [27], suggesting the need to evaluate complex abilities, such as working memory and simple memory.
Limitations
This study had several limitations. First, we defined MCI using the MMSE-only scores. Although it is recommended that MCI be evaluated using the MoCA [11, 28], previous studies have shown that the discriminative ability (AUC) of the MoCA and MMSE for MCI were 0.88 and 0.78, respectively. This indicates that the MMSE is slightly inferior to the MoCA; however, the MMSE has a certain degree of accuracy [29]. Moreover, we did not perform a definitive diagnosis of MCI in this study. Therefore, healthy individuals may have been included among the patients with MCI. However, because the purpose of this study was screening and not to make a definitive diagnosis, we used a practical measure to evaluate MCI. The incidence of MCI in this study was 35.2 per 1000 person-years, which is a reasonable value compared to that in previous studies (8.5 and 76.8 per 1000 person-years) [21]. Therefore, the evaluation of MCI in this study appears to have sufficient validity. Second, this study was a venue-invitation survey, which may have introduced selection bias, and the participants had relatively high levels of education and mental and physical functioning. Therefore, the results of this study should be interpreted with caution. Further studies with a wider range of participants are warranted. Finally, the number of outcome events in this study was 32; the number of explanatory variables entered was nine; and the number of events per variable (EPV) was 3.56. In the logistic regression model, previous studies have highlighted that when the EPV is <10, the confidence interval of the regression coefficient becomes wide, affecting the reliability of the results [30]. However, the confidence intervals obtained in this study were sufficiently narrow. Simulations in previous studies have shown that the effect of a low EPV is even smaller when the independent variable is a continuous number [31]. Therefore, in this study, the effect of a low EPV was minimal, and the results obtained were considered reliable. Finally, the study did not include internal validation because of the small number of MCI cases, which may have resulted in overfitting. In the future, increasing the number of participants and validating the results using different datasets may be necessary.
Conclusion
This study examined the ability of CompBased-CAT to predict the occurrence of MCI within two years. The results showed good accuracy, with an AUC, sensitivity, and specificity of 0.79, 0.76, and 0.75, respectively, indicating that CompBased-CAT can detect MCI, which is an early stage of dementia. We hope the CompBased-CAT will be used in future community health checkups and events for older adults.
Acknowledgments
We are grateful to the local authorities of the Itabashi Ward and the individuals who participated in this study.
References
- 1.
International AsD. World Alzheimer report. Alzheimer’s Disease International; 2018.
- 2. Cheng ST. Dementia caregiver burden: a Research update and critical analysis. Curr Psychiatry Rep. 2017;19: 64. pmid:28795386.
- 3. Wimo A, Guerchet M, Ali GC, Wu YT, Prina AM, Winblad B, et al. The worldwide costs of dementia 2015 and comparisons with 2010. Alzheimers Dement. 2017;13: 1–7. pmid:27583652.
- 4. Petersen RC, Doody R, Kurz A, Mohs RC, Morris JC, Rabins PV, et al. Current concepts in mild cognitive impairment. Arch Neurol. 2001;58:1985–1992. pmid:11735772
- 5. Livingston G, Sommerlad A, Orgeta V, Costafreda SG, Huntley J, Ames D, et al. Dementia prevention, intervention, and care. Lancet. 2017;390: 2673–2734. pmid:28735855
- 6. Valls-Pedret C, Sala-Vila A, Serra-Mir M, Corella D, de la Torre R, Martínez-González MÁ, et al. Mediterranean diet and age-related cognitive decline: A randomized clinical trial. JAMA Intern Med. 2015;175: 1094–1103. pmid:25961184.
- 7. Kelly ME, Loughrey D, Lawlor BA, Robertson IH, Walsh C, Brennan S. The impact of exercise on the cognitive functioning of healthy older adults: a systematic review and meta-analysis. Ageing Res Rev. 2014;16: 12–31. pmid:24862109.
- 8. Carlson MC, Saczynski JS, Rebok GW, Seeman T, Glass TA, McGill S, et al. Exploring the effects of an “everyday” activity program on executive function and memory in older adults: Experience Corps. Gerontologist. 2008;48: 793–801. pmid:19139252.
- 9. Bunn F, Goodman C, Sworn K, Rait G, Brayne C, Robinson L, et al. Psychosocial factors that shape patient and carer experiences of dementia diagnosis and treatment: a systematic review of qualitative studies. PLOS Med. 2012;9: e1001331. pmid:23118618.
- 10. O’Bryant SE, Humphreys JD, Smith GE, Ivnik RJ, Graff-Radford NR, Petersen RC, et al. Detecting dementia with the Mini-Mental State Examination in highly educated individuals. Arch Neurol. 2008;65: 963–967. pmid:18625866
- 11. Nasreddine ZS, Phillips NA, Bédirian V, Charbonneau S, Whitehead V, Collin I, et al. The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc. 2005;53: 695–699. pmid:15817019.
- 12. Nieuwenhuis-Mark RE. The death knoll for the MMSE: has it outlived its purpose? J Geriatr Psychiatry Neurol. 2010;23: 151–157. pmid:20231732.
- 13. Takahashi J, Kawai H, Suzuki H, Fujiwara Y, Watanabe Y, Hirano H, et al. Development and validity of the Computer-Based Cognitive Assessment Tool for intervention in community-dwelling older individuals. Geriatr Gerontol Int. 2020;20: 171–175. pmid:31916344.
- 14. Kawai H, Ihara K, Kera T, Hirano H, Fujiwara Y, Tanaka M, et al. Association between statin use and physical function among community-dwelling older Japanese adults. Geriatr Gerontol Int. 2018;18: 623–630. pmid:29278297.
- 15. Mitchell AJ. A meta-analysis of the accuracy of the Mini-Mental State Examination in the detection of dementia and mild cognitive impairment. J Psychiatr Res. 2009;43: 411–431. pmid:18579155
- 16. Inagaki H, Ito K, Sakuma N, Sugiyama M, Okamura T, Awata S. Reliability and validity of the simplified. Japanese version of the WHO-Five Well-being Index (S-WHO-5-J). Nihon Koshu Eisei Zasshi. 2013;60: 294–301.
- 17. Koyano W, Shibata H, Nakazato K, Haga H, Suyama Y. Measurement of competence: reliability and validity of the TMIG Index of Competence. Arch Gerontol Geriatr. 1991;13: 103–116. pmid:15374421
- 18. De Roeck EE, De Deyn PP, Dierckx E, Engelborghs S. Brief cognitive screening instruments for early detection of Alzheimer’s disease: a systematic review. Alzheimers Res Ther. 2019;11: 21. pmid:30819244.
- 19. Taniguchi Y, Kitamura A, Murayama H, Amano H, Shinozaki T, Yokota I, et al. Mini-Mental State Examination score trajectories and incident disabling dementia among community-dwelling older Japanese adults. Geriatr Gerontol Int. 2017;17: 1928–1935. pmid:28296176.
- 20. Watanabe Y, Arai H, Hirano H, Morishita S, Ohara Y, Edahiro A, et al. Oral function as an indexing parameter for mild cognitive impairment in older adults. Geriatr Gerontol Int. 2018;18: 790–798. pmid:29380503.
- 21. Luck T, Luppa M, Briel S, Riedel-Heller SG. Incidence of mild cognitive impairment: a systematic review. Dement Geriatr Cogn Disord. 2010;29: 164–175. pmid:20150735.
- 22. Shimada H, Makizako H, Park H, Doi T, Lee S. Validity of the National Center for Geriatrics and Gerontology-Functional Assessment Tool and Mini-Mental State Examination for detecting the incidence of dementia in older Japanese adults. Geriatr Gerontol Int. 2017;17: 2383–2388. pmid:28620925.
- 23. Na KS. Prediction of future cognitive impairment among the community elderly: A machine-learning based approach. Sci Rep. 2019;9: 3335. pmid:30833698
- 24. Bäckman L, Jones S, Berger AK, Laukka EJ, Small BJ. Cognitive impairment in preclinical Alzheimer’s disease: a meta-analysis. Neuropsychology. 2005;19: 520–531. pmid:16060827.
- 25. Palmer K, Bäckman L, Winblad B, Fratiglioni L. Detection of Alzheimer’s disease and dementia in the preclinical phase: population based cohort study. BMJ. 2003;326: 245. pmid:12560271
- 26.
Wechsler D. Wechsler Adult Intelligence Scale. 4th ed. San Antonio, Texas: Pearson; 2008.
- 27. White DA, Murphy CF. Working memory for nonverbal auditory information in dementia of the Alzheimer type. Arch Clin Neuropsychol. 1998;13: 339–347. pmid:14590613
- 28. Fujiwara Y, Suzuki H, Yasunaga M, Sugiyama M, Ijuin M, Sakuma N, et al. Brief screening tool for mild cognitive impairment in older Japanese: validation of the Japanese version of the Montreal Cognitive Assessment. Geriatr Gerontol Int. 2010;10: 225–232. pmid:20141536.
- 29. Pinto TCC, Machado L, Bulgacov TM, Rodrigues-Júnior AL, Costa MLG, Ximenes RCC, et al. Is the Montreal Cognitive Assessment (MoCA) screening superior to the Mini-Mental State Examination (MMSE) in the detection of mild cognitive impairment (MCI) and Alzheimer’s disease (AD) in the elderly? Int Psychogeriatr. 2019;31: 491–504. pmid:30426911.
- 30. Peduzzi P, Concato J, Kemper E, Holford TR, Feinstein AR. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol. 1996;49: 1373–1379. pmid:8970487.
- 31. Vittinghoff E, McCulloch CE. Relaxing the rule of ten events per variable in logistic and cox regression. Am J Epidemiol. 2007;165: 710–718. pmid:17182981.