A new computerized assessment battery for cognition (C-ABC) to detect mild cognitive impairment and dementia around 5 min

This study aimed to develop a new computerized assessment battery for cognition (C-ABC) to detect mild cognitive impairment (MCI) and dementia. We performed C-ABC in subjects with dementia (n = 422), MCI (n = 145), and normal cognition (NC; n = 574), and analyzed by age stratum (50s, 60s, and 70–85 years). To distinguish MCI from NC, the C-ABC total combined score, which were calculated by dividing the C-ABC total score by the C-ABC required time, revealed the best area under the curves (AUC) at 0.838 and 0.735 in the 50s and 60s age groups, respectively; notably, this entire procedure took approximately 5 min. To distinguish dementia from NC and MCI, the partial items of C-ABC (items 3 + 6 combined score) revealed the best AUCs at 0.910, 0.874, and 0.882 in the 50s, 60s, and 70–85 age groups, respectively. Furthermore, the items 3 + 6 combined score established the best AUC at 0.794 in the 70–85 age group to distinguish MCI from NC; this entire procedure took around 2 min. Hence, this study suggests that C-ABC could be a useful tool for detecting dementia or MCI in a short time.


Introduction
The number of patients with dementia has markedly increased as the population has aged around the world [1]. Thus, the importance of early detection of individuals with mild cognitive impairment (MCI) and dementia is now widely accepted. The computerized cognitive test battery is superior over pencil-and-paper based classical cognitive test in terms of precision measurement of required time for cognitive tests and no need of trained medical staff [2]. Additionally, web-based testing provides the potential for large-scale screening of populations for cognitive function [3]. To detect MCI, the administration time for these computerized cognitive tests usually ranges from 10 to 30 minutes [4]. Although there is a lot of computerized cognitive test batteries, it was reported as difficult to select a test that would be most suitable for detecting dementia, and one that would be most suitable for detecting MCI, respectively [4]. Therefore, this study aimed to develop a sensitive and easily administered computerized cognitive test battery that could detect not only dementia but also MCI in a short time. In addition, we aimed to validate a new computerized assessment battery for cognition (C-ABC) to detect MCI and dementia with high sensitivity or specificity in older adults.

Study population
We examined 758 consecutive patients who visited our memory clinic and underwent C-ABC (S1 Fig). Of these, we excluded 39 patients with psychiatric disorders (n = 36), such as depression, and disturbance of consciousness (n = 3). In addition, 18 patients were withdrawn from C-ABC because of hearing impairment (n = 9), difficulty of comprehension (n = 7), and visual impairment (n = 2). The remaining 701 patients were enrolled in this study, including patients with dementia (n = 422), MCI (n = 145), and normal cognition (NC; n = 134) (S1 Fig). The diagnoses of dementia and MCI were based on the guidelines of the Diagnostic and Statistical Manual of Mental Disorders, third edition, revised (DSM-III-R) [5] and the International Working Group on general criteria for MCI [6], respectively. The MCI criteria state that: 1) the individual should be judged as unhealthy using modalities other than those used to fulfill the DSM III-R dementia criteria; 2) the individual's functional activities are mainly preserved or at least minimally impaired; and 3) the individual should have evidence of cognitive decline, either by self-assessment and/or by informant report in conjunction with deficits in objective cognitive tasks [6]. Among participants without dementia, a clinical dementia rating (CDR) [7] comprehensive score of 0.5 was used as the objective cognitive impairment value to denote cognitive and functional impairment consistent with MCI. Besides, we enrolled 440 individuals with NC from the Ishikawa Brain Imaging Study (S1 Fig) that was an imaging study aiming to develop imaging biomarkers for early and objective assessment of Alzheimer's disease (AD) and other forms of neurodegenerative diseases using positron emission tomography and magnetic resonance imaging [8]. Of note, all subjects with NC, enrolled from our memory clinic and the Ishikawa Brain Imaging Study, were examined with the Mini-Mental State Examination (MMSE) [9] and CDR to assess cognitive profiles; MMSE cutoff point � 24 of 30 and a CDR score of 0 suggested cognitively normal state [8,9]. Additionally, subjects with NC had no history of psychiatric or neurological diseases. Furthermore, all subjects underwent medical screening, including a questionnaire about medical history and general and neurological examinations.
The age of the subjects ranged 23-95 years. As the cognitive function is susceptible to age, we analyzed the data by age stratum. We excluded patients aged <50 and >85 and analyzed only those aged 50-85 years, because only 1 patient with MCI aged <50 years and only 2 with NC aged >85 years. The remaining patients were divided into the following 3 age groups: 50s, 60s, and 70-85 years (70-85 group). Then, the dementia, MCI, and NC groups were created that matched age, education period, and gender by random sampling using SPSS software (version 23; SPSS Inc., Chicago, IL). After random sampling, there were 336 subjects with dementia, 137 with MCI, and 367 with NC (S1 Fig). Of 336 patients with dementia, 249 had AD, 23 had vascular dementia (VaD), 12 had mixed dementia (AD and VaD), 13 had dementia with Lewy bodies (DLB), 7 had frontotemporal dementia (FTD) and 32 had undetermined dementia. The diagnosis of AD was based on the criteria of the National Institute of Neurological and Communicative Disorders and stroke and the Alzheimer's Disease and Related Disorders Association [10], VaD on the criteria of the National Institute of Neurological Disorders and Stroke and Association and Internationale pur la Rechrche et l'Enseignement en Neurosciences [11], DLB on the criteria of the third report of the DLB Consortium [12], and FTD on the diagnostic criteria of FTD [13].
This study was approved by the Medical Ethics Review Board of Kanazawa University (Kanazawa, Japan; approval number 1845). Written informed consent was obtained from all patients or their legal representatives.

Computerized cognitive assessment battery
The C-ABC procedure was described to all subjects before the test administration. We usually examined C-ABC on a different day from MMSE, within a month. When the subjects were examined with C-ABC, a psychologist stayed to help them as necessary. All questions and instructions were presented in both a text on the PC screen (horizontal 80 cm × vertical 60 cm) and a verbal description through headphones (S2 Fig). All subjects were instructed to touch the touchscreen when they answer the C-ABC. The C-ABC comprised eight items, including sensorimotor skill, attention, orientation, immediate memory, and an arithmetic problem ( Table 1).
The C-ABC total score ranges from 0 to 40 points. The time required for C-ABC (total required time; s) was automatically measured for each subject. The C-ABC total and each item Touching a moving target The circle target was presented in different locations on the screen, one at a time, and the subject was asked to touch the circle target as quickly as possible. 10 2 The digits order Nine digits (1)(2)(3)(4)(5)(6)(7)(8)(9) were presented at random positions on the screen and the subject was asked to touch the digits in sequential order as quickly as possible. 9 3

Time orientation
The subject was asked to choose the date of today (day, month, year, Japanese era name, and day of the week) from a list of candidates on the screen.

4
The letters-recognition memory test Four Japanese letters of hiragana ("ri", "na", "ku", and "me") with meaningless relationships were presented on the screen for 5 seconds. Then, the subject was asked to select the recognized four letters from the Japanese syllabary table.

5
The numbers-recognition memory test Three numbers without serial number were presented one by one on the screen. After 5 s, the subject was asked to select the three numbers in the correct order from the number plate.

6
The figures-recognition memory test Four figures with different conditions in color and shape were presented on the screen. After 5 s, the subject was asked to select the recognized 4 figures from a set of 12 candidates.

7
The arithmetic problem A shopping story was presented on the screen. The subject was asked about the total number of goods purchased from two stores. Then, the subject was asked to select the correct number from a set of four options.

PLOS ONE
combined scores were evaluated by dividing the C-ABC total and each item score by total and each item required time (s) and multiplying by 1000, respectively.

Statistical analyses
First, we compared the clinical characteristics, MMSE score, C-ABC total score, each item score, required time, C-ABC total combined score, and each item combined score between NC, MCI, and dementia groups, using the Kruskal-Wallis test or χ 2 test for each of the 50s, 60s and 70-85 groups. A correlation between C-ABC combined score and MMSE score was analyzed using Pearson's correlation test.
Next, accuracies to diagnose dementia (dementia vs. NC and MCI) and MCI (MCI vs. NC) for the C-ABC total combined score, each item combined score, and MMSE were assessed by using the receiver-operating characteristic (ROC) analysis. Generally, a cognitive test that can distinguish dementia or MCI with high sensitivity and high specificity is the best. In this study, the optimal cutoff value (OCV) was defined as the cutoff point with the maximum value of the sum of the value of sensitivity and specificity. In addition, the ROC analyses revealed that items 3 (time orientation) and 6 (the figures-recognition memory test) were the best parameters among the item combined scores. We also performed the multivariate logistic regression analysis to evaluate which item combined score had large effect to distinguish dementia or MCI. We made the items 3 + 6 combined score which was calculated by dividing sum of items 3 and 6 scores by sum of items 3 and 6 required time and multiplying by 1000. The ROC analyses of the C-ABC total, each item, and items 3 + 6 combined scores revealed no ideal cutoff score with which both sensitivity and specificity were �0.9 in any age group.
Then, to distinguish MCI from NC and dementia from NC and MCI with high sensitivity, we defined the minimum score with sensitivity �0.9 as "sensitivity �0.9". When C-ABC is performed as a screening test at hospitals followed by cognitive tests by trained medical staff, the cutoff score with high sensitivity should be used to decrease false negatives as much as possible.
Regarding specificities, we defined the maximum score with specificity �0.9 as "specificity �0.9" to distinguish MCI from NC and dementia from NC and MCI with high specificity. Older adults can perform C-ABC at home or any public facility other than hospitals because trained medical staffs are not needed. If older adults do self-inspection to detect dementia or MCI without following cognitive tests performed by trained medical staff, false positives seem to increase mental, time, and financial burdens. It must be important to set a high specificity cutoff value to decrease false positives.
In addition, we performed ten-fold cross-validation of ROC for the C-ABC combined score, by using all age groups data.
All data are presented as mean (SD) unless otherwise specified. We considered P < 0.05 as statistically significant. The ROC analyzes were performed with EZR (Saitama Medical Center, Jichi Medical University, Saitama, Japan), which is a graphical user interface for R (The R Foundation for Statistical Computing, Vienna, Austria). More precisely, it is a modified version of R commander designed to add statistical functions frequently used in biostatistics [14]. Other statistical analyses and random sampling were performed using the SPSS software package (version23; SPSS Inc., Chicago, IL).

Subjects' characteristics
The demographic and clinical characteristics of subjects are shown in Table 2.

PLOS ONE
We observed no significant differences in gender ratio, mean age, and education period between dementia, MCI, and NC in the 60s and 70-85 groups. In the 50s group, the proportion of females was significantly lower in the MCI group than that in the dementia and NC groups.

C-ABC total combined score
In all age groups, the mean C-ABC required time was approximately 5 min for dementia, MCI, and NC. The C-ABC combined score significantly correlated with the MMSE score (r = 0.753, P < 0.001; Fig 1).
In addition, the C-ABC combined score was significantly lower in MCI and dementia than in NC in all age groups (Fig 2A-2C and S1 Table).
Furthermore, the C-ABC combined score was significantly lower in dementia than in MCI in the 60s and 70-85 groups (Fig 2B and 2C and S1 Table).
When the C-ABC combined score was used, there was no ideal cutoff point with which both sensitivity and specificity were �0.9. The sensitivities/specificities at the OCV of the C-ABC combined score to distinguish dementia from MCI and NC were 0.84/0.84, 0.90/0.74, and 0.85/0.67 for the 50s, 60s, and 70-85 groups, respectively (Fig 3A-3C and Table 3).
Additionally, we performed ten-fold cross-validation of ROC for the C-ABC combined score using all age groups. The mean sensitivities/ specificities at the OCV of the C-ABC combined score to distinguish dementia from MCI and NC and MCI from NC were 0.91/ 0.63 and 0.80/0.59, respectively.

PLOS ONE
Each item score and items 3 + 6 combined score Among items, the mean score and percentage of full score of item 3 (time orientation) were very low in dementia than that in NC and MCI (S1 Table). As item 6 (the figures-recognition memory test) was relatively difficult, only 72.5% of NC subjects in the 50s group could answer all correctly (S1 Table). Then, the mean score and percentage of full score of item 6 were very low in MCI and dementia than that in NC in all age groups, and the required time was prolonged, especially in the 50s group. Almost all item combined scores were significantly lower in dementia than that in NC in all age groups and were significantly lower in dementia than that in MCI in the 60s and 70-85 groups (S2 Table). Among items, only item 6 combined score was significantly lower in MCI than in that NC in all age groups (S2 Table). Multivariate logistic regression analyses showed that item 3 combined score had the most effect for distinguishing dementia, and item 6 combined score had the second most effect (S3 Table). Additionally, for distinguishing MCI, both of item 3 and 6 combined scores had almost equally the most effect in the multivariate logistic regression analysis (S2 Table). The diagnostic accuracy of each item combined score is shown in Table 3. In this study, we could not find the ideal cutoff point with which both sensitivity and specificity were �0.9 among item combined scores (Table 3).
Regarding the area under the curve (AUC) using the score of each item combined score for C-ABC, item 3 combined score exhibited the maximum AUC to distinguish dementia from MCI and NC in all age groups, and MCI from NC in the 60s and 70-85 groups (Table 3). Besides, item 6 combined score was the best parameter with the maximum AUC to distinguish

PLOS ONE
MCI from NC in the 50s group (Table 3). Therefore, we made the items 3 + 6 combined score. The mean required time to perform items 3 and 6 was approximately 2 min for dementia, MCI, and NC in each age group. Moreover, the items 3 + 6 combined score significantly correlated with the MMSE score (r = 0.740, P < 0.0001; S3 Fig). In addition, the items 3 + 6 combined score was significantly lower in MCI and dementia than that in NC in all age groups (S4A- S4C  Fig and S1 Table). Besides, the items 3 + 6 combined score was significantly lower in dementia than that in MCI in the 60s and 70-85 groups (S4B and S4C Fig and S1 Table). When the items  3 + 6 combined score was used, no ideal cutoff point was found with which both sensitivity and specificity were � 0.9. Comparison of the diagnostic accuracies between C-ABC total combined score and items 3 + 6 combined score revealed that the items 3 + 6 combined score exhibited a marginally better AUC to distinguish dementia from MCI and NC in all age groups (Fig 3A-3C and Table 3), and the items 3 + 6 combined score was better for distinguishing MCI from NC in the 70-85 group. In the 50s and 60s groups, however, the C-ABC total combined score was better to distinguish MCI from NC (Fig 3A-3C and Table 3).

Detection of MCI/ dementia with sensitivity �0.9
We analyzed the cutoff value for sensitivity �0.9. When the C-ABC total combined score was used and the cutoff points of sensitivity �0.9 were set to 138, 132, and 123 for the 50s, 60s, and 70-85 groups, respectively, the sensitivities to distinguish dementia from MCI and NC were 0.91, 0.90, and 0.90, respectively (Table 4). Likewise, when the cutoff points of sensitivity �0.9 to distinguish MCI for NC were set to 147, 146, and 138 for the 50s, 60s, and 70-85 groups, respectively, the sensitivities were 1.00, 0.91, and 0.90, respectively (Table 4). In addition, when using the MMSE score similarly, the cutoff points of sensitivity �0.9 were 30, 30, and 29 for the 50s, 60s, and 70-85 groups, respectively, and the sensitivities for detecting MCI were 1.00, 1.00, and 0.99, respectively (Table 4).
When the items 3 + 6 combined score was used, the cutoff points of sensitivity �0.9 were set to 68.23, 65.62, and 57.85 for the 50s, 60s, and 70-85 groups, respectively, and the sensitivities to distinguish dementia from MCI and NC were 0.92, 0.90, and 0.90, respectively (S2 Table). Furthermore, when the cutoff points of sensitivity �0.9 to distinguish MCI for NC were set to 80.86, 80.07 and 69.74, for the 50s, 60s and 70-85 groups, respectively, the sensitivities were 1.00, 0.91 and 0.90 respectively (S2 Table).

Detection of MCI/ dementia with specificity �0.9
We also analyzed the cutoff value for specificity �0.9. When the C-ABC combined score was used and the cutoff points of specificity �0.9 were set to 111, 107, and 96 for the 50s, 60s, and 70-85 groups, respectively, the specificities to distinguish dementia from MCI and NC were 0.91, 0.90, and 0.90, respectively (Table 4). Similarly, when the cutoff points of specificity �0.9 to distinguish MCI for NC were set to 114, 110, and 101, for the 50s, 60s, and 70-85 groups, respectively, the specificities were 0.93, 0.90, and 0.91, respectively (Table 4).
When the items 3 + 6 combined score was used, and the cutoff points of specificity �0.9 were set to 48.82, 36.17, and 36.04 for the 50s, 60s, and 70-85 groups, respectively, the  Table).

Discussion
The major findings of this study were as follows: (i) the C-ABC only took approximately 5 min; (ii) the C-ABC total combined score cutoff points with high sensitivities or specificities could be set according to the purpose of usage; and (iii) the items 3 + 6 combined score, which

PLOS ONE
took only 2 min, displayed better diagnostic accuracies to detect dementia compared with the C-ABC combined score.
Regarding comparison with MMSE, the C-ABC combined score and the items 3 + 6 combined score highly correlated with MMSE scores. Some cognitive domains of C-ABC, such as attention, orientation, and recognition memory, are similar to MMSE. Regarding sensitivities, setting the cutoff value of 30 was essential to attain the sensitivity of �0.9 in MMSE. As full score of MMSE is 30 points, a cutoff of 30 points is irrelevant to taking the test, and detecting MCI by MMSE with high sensitivity would be difficult. It was reported that MMSE may not detect MCI [15] or mild dementia [16].
The C-ABC total combined score using a high sensitivity cutoff value �0.9 is appropriate for screening tests, followed by cognitive tests by trained medical staff. If C-ABC is performed outside the hospital or health examination in the absence of following cognitive tests, it is essential to set a high specificity cutoff value to decrease false positives. In such cases, false negatives would increase; however, repeated C-ABC probably will be able to detect MCI and dementia early.

PLOS ONE
In addition, C-ABC can be completed within a short time (approximately 5 min); the administration time of C-ABC was shorter than that of MMSE (approximately 10 min for older person [15]). In addition, the AUC of the item 3 + 6 combined score in our study was higher than that of the C-ABC combined score to detect dementia in all age groups. Among items, the mean score and percentage of full score of item 3 (time orientation) were very low in dementia, and those of item 6 (the figures-recognition memory test) were very low in MCI and dementia. Although patients with dementia usually have disorientation and severe memory disturbance, those with MCI and NC rarely have both. Thus, item 3 that asks time orientation and item 6 that requires to memory 4 figures in detail would be beneficial in distinguishing dementia from MCI and NC. In our study, items 3 + 6 were performed in only around 2 min. With regard to computerized cognitive battery, the Cogstate Brief Battery, which comprises four cognitive tasks and was reported to be able to distinguish patients with MCI from healthy older adults requires 10 min [17]. A brief computerized paired-associate test (the Miami Test of Semantic Interference and Learning) to detect MCI requires around 9 min [18]. As computerized cognitive test batteries, such as C-ABC, could be performed on numerous people simultaneously in a short time, it is beneficial for screening both MCI and dementia.
This study has some limitations. First, the sample size was relatively small. Thus, further longitudinal studies examining a larger sample size with various cognitive impairments are needed to measure the reproducibility and broad application of C-ABC. Second, the items 3 + 6 combined score displayed better accuracies to distinguish dementia compared with the C-ABC combined score. However, we did not examine measurements using only items 3 and 6. Thus, additional studies using only items 3 and 6 are warranted to elucidate the utility of items 3 and 6.
In conclusion, we developed the new C-ABC that could detect patients with MCI and dementia in older adults with high sensitivities or specificities in approximately 5 min. Furthermore, the items 3 + 6 combined score could detect patients with dementia in only around 2 min.
Supporting information S1 Fig. The flowchart of the subjects. We excluded subjects aged <50 and >85, and analyzed only those 50-85 years, because only 1 subject with MCI aged <50 years and only 2 with NC aged >85 years. We excluded 194 subjects aged <50 years and 2 aged >85 years from the NC group ( � 1). We excluded 1 subject aged <50 years and 4 aged >85 years from the MCI group ( � 2). We excluded 6 subjects aged <50 years and 29 aged >85 years from the dementia group ( � 3). In addition, the dementia, MCI, and NC groups were created that matched age, education period, and gender by random sampling using SPSS software (version 23; SPSS Inc., Chicago, IL). Consequently, we excluded 11, 3, and 51 subjects from the NC ( � 1), MCI ( � 2), and dementia ( � 3) groups, respectively. Overall, we excluded 207 subjects from the NC group, 8 from the MCI group, and 86 from the dementia group, and finally examined 367 subjects as the matched NC group, 137 as the matched MCI group, and 336 as the matched dementia group. C-ABC, computerized assessment battery for cognition; MCI, mild cognitive impairment; NC, normal cognition.  Table. The mean score and percentage of full score, required time, combined score � for each item, all items, and items 3 + 6 in subjects with NC, MCI, and dementia for the 50s, 60s, and 70-85 groups.