Cut-off Scores of a Brief Neuropsychological Battery (NBACE) for Spanish Individual Adults Older than 44 Years Old

The neuropsychological battery used in Fundació ACE (NBACE) is a relatively brief, and easy to administer, test battery that was designed to detect cognitive impairment in the adulthood. The NBACE includes measures of cognitive information processing speed, orientation, attention, verbal learning and memory, language, visuoperception, praxis and executive functions. The aim of the present study was to establish the cut-off scores for impairment for different levels of age and education that could be useful in the cognitive assessment of Spanish subjects who are at risk for cognitive impairment, especially dementia. Data from 1018 patients with a mild dementia syndrome, and 512 cognitively healthy subjects, older than 44 years, from the Memory Clinic of Fundació ACE (Barcelona, Spain) were analyzed. In the whole sample, cut-off scores and sensitivity/specificity values were calculated for six conditions after combining 3 age ranges (44 to 64; 65 to 74; and older than 74 years old) by 2 educational levels (until Elementary school; and more than Elementary school). Moreover, general cut-offs are reported for Catalan and Spanish speakers. The results showed that most of NBACE tests reached good sensitivity and specificity values, except for Ideomotor praxis, Repetition and Verbal Comprehension tests, which had a ceiling effect. Word List Learning from the Wechsler Memory Scale-III and Semantic Verbal Fluency were the most useful tests to discriminate between cognitively healthy and demented subjects. The NBACE has been shown to be a useful tool able to detect cognitive impairment, especially dementia, in older than 44 years Spanish persons.


Introduction
Neuropsychological assessment is critical for the diagnosis of clinical dementia syndromes or other forms of cognitive impairment [1,2]. Subtle cognitive deficits generally cannot be detected by the screening tools frequently used by primary care physicians (e.g., Mini-Mental State Examination, MMSE) [3]. So, there is a need for brief neuropsychological batteries, covering a wide range of cognitive functions, able to detect subtle deficits in clinical settings.
In the clinical practice, raw neuropsychological test scores are adjusted for socio-demographic factors in order to facilitate an accurate interpretation of test performance. The scores can be adjusted in different ways, such as using normative data covering a range of tests [4,5,6,7] or normative data from specific populations, such as the community-dwelling elders [8,9].
In Spain, tests have been usually translated from English versions and adapted using foreign population normative data. To aid this process, projects like the Spanish Multicenter Normative Studies (NEURONORMA) have been adapting English language tests for the Spanish-speaking population [10,11,12,13,14]. A few neuropsychological screening batteries, adapted for Spanishspeaking individuals, have also been adapted to include age and education adjusted normative values [15,16,17,18,19]. All of these tests were standardized by administering them to a representative sample of cognitively healthy individuals to establish normative scores.
Neuropsychologists generally use a threshold of 1.5 standard deviation units below the performance mean of healthy population as an indication of impaired function [20,21]. However, this strategy has the limitation that the classification of ''abnormal'' depends on the characteristics of the normative sample. Further-more, the cut-off scores calculated using this strategy have a mathematical meaning that sometimes does not meet the clinical need. This psychometric approach does not necessarily provide an empirical estimation of the relevant cut-off value for abnormal performance. To establish the limits between preserved and impaired function, it is important to include patients with a known pathological condition (e.g. patients with dementia), which allows the calculation of sensitivity (probability to detect a dementia case) and specificity (probability to detect a non-dementia case) values for a test score.
The NBACE is a brief neuropsychological battery developed by Fundació ACE for use in the Memory Clinic as a component of the diagnostic process of cognitive impairment and dementia [19]. It assesses the cognitive functions relevant to the diagnosis of   [19,31]. The aim of the present study was to establish the cut-off scores for impairment for different levels of age and education that could be useful in the cognitive assessment of Spanish subjects who are at risk for cognitive impairment, especially dementia.

Participants
Fundació ACE, Institut Català de Neurociències Aplicades, is a nonprofit Alzheimer's center that provides diagnostic, treatment and patient management services to the Catalan Public Health Service (Xarxa Hospitalària d'Utilització Pública (XHUP)). The patients are usually referred to the Memory Clinic of Fundació ACE by primary care physicians or medical specialists because the patients, their family, or their physician felt that they could have a memory problem [31].
The data included for the purpose of the present study were drawn from individuals older than 44 who were examined at Fundació ACE between September 2008 and January 2013. To establish cut-off scores, data from 1018 patients with a ''Mild Dementia Syndrome'' (MDS) and 512 cognitively healthy subjects (HS) were analysed (Table 1).
General inclusion criteria were predefined as follows: Subjects older than 44 years old, without severe auditory or visual abnormalities that could affect performance on neuropsychological tests. With regard to MDS group, inclusion criteria were: A diagnosis of MDS, with a mild severity of dementia determined by a total Clinical Dementia Rating (CDR; [32]) of 1 and a Mini-Mental State Examination (MMSE; [27]) higher than 19. With regard to HS group, inclusion criteria were: A diagnosis of ''without cognitive impairment'', a CDR of zero, preserved scores on the Memory test of the 7 Minute Screen test [33] ($12), on the MMSE (($24) and on the Spanish version of the Clock Test [34] ($5), and they had no evidence of functional impairment due to cognitive decline, with a score lower than 4 on the Blessed Dementia Rating Scale (BDRS) [35,36] and a score lower than 17 in the Rapid Disability Rating Scale-2 (RDRS-2) [36,37]).    Those subjects younger than 45 years old, with analphabetism or with severe auditory or visual abnormalities that could interfere on neuropsychological test performances were excluded from the study.
Written informed consent was obtained from all participants, and their caregivers in case of dementia, prior to any evaluation. The study was approved by the Fundació ACE Research Ethics Committee.

Measures
All of the participants received an extensive clinical evaluation including a neurological history and examination, a semistructured psychosocial interview conducted by a social worker and the NBACE assessment.
All of the neuropsychological testing was carried out in the Diagnostic Unit of Fundació ACE by one of the three neuropsychologists (MA, AE, and GVJ), and it was completed in the     Table 2, score ranges are detailed.

Procedure
The preliminary diagnosis of each individual was initially made by the neurologist based only on the clinical history and exam, and not on the NBACE results. The initial classification was later reviewed by the study team, including neurologists, neuropsychologists and social workers, at a consensus diagnostic conference. The dementia diagnosis was mainly based on the impairment on the activities of daily living. Functionality was assessed by the BDRS [35] and the RDRS-2 [37], both of them validated in Spanish population [36]. A score higher than 4 on the BDRS, and a score higher than 16 on the RDRS-2, meant that the subject was dependent on their activities of daily living [36]. Once the dementia was diagnosed, NBACE performances profile contributed to specify the dementia type.

Statistical Analysis
Statistical analysis was carried out using SPSS 20 (SPSS Inc., Chicago, IL). T-tests were used to compare age and MMSE scores between groups, and Chi-square for education and gender comparisons. A sample of 1018 patients with a MDS and 512 HS was used to calculate cut-off scores and sensitivity/specificity values for six conditions after combining 3 age ranges (44 to 64; 65 to 74; and older than 74 years) by 2 educational levels (through  elementary school (#8 years); and more than elementary school (.8 years)) (see Table 3). Multiple imputation methods were used to account for any missing test data, which was ,0.3% of the total. For the whole sample, and for each language subgroup (Catalan and Spanish), cut-off points were established by calculating the sensitivity and specificity for each test score. Receiver operating characteristic (ROC) analysis was used to calculate the optimal cut-off value between MDS and HS groups. Moreover, to calculate the sensitivity and specificity of each cut-off score, an area under the curve (AUC) analysis was also carried out including its confidence interval 95%.
Our goal was to obtain an AUC greater than 0.75 for each test variable and for the most of 6 (age by education) conditions, sensitivity and specificity values were greater than 0.75. For those variables where sensitivity and specificity values were lower than 0.75, a unique cut-off score was calculated for the entire sample adjusting by age and educational level.

Results
MDS patients were older and had a lower educational level than HS (See Table 1). For the purpose of the present study, it was not a problem, because cut-offs were reported after combining 3 age ranges (45-64, 65-74 and .74) by 2 educational levels (# Elementary school and.Elementary school) (Tables 4-12), and global cut-offs were adjusted by age and education (Table 13).
The AUC, means and SD of the NBACE test variables are detailed in Table 2. When the AUC was greater than 0.75, the cut-offs were reported after identifying the cut-off score that yielded approximately equal sensitivity and specificity values. As shown in Table 2, the measures that best discriminated between MDS and HS groups were the Word List Verbal Learning and Semantic Verbal Fluency tests.
Tables 4-12 provide the age and education adjusted cut-off scores for verbal learning and long-term memory (including recognition memory and forgetting), language and executive functioning. However, Global orientation, Digit span Forward and Backwards, Block Design, Imitation and Global praxis, Poppelreuter's test, Luria's Clock test, 15-OT and Automatic Inhibition SKT subtest (number of errors) showed sensitivity and specificity values for several conditions lower than 0.75. In these cases, a unique cut-off score was calculated for the entire sample adjusting by age and educational level.
The Digit Span Forward measure of WAIS-III (AUC = 0.69) did not meet the established AUC threshold. However, a valid global cut-off score was obtained, with good sensitivity and specificity values, after adjusting by age and education. Ideomotor praxis, Repetition and Verbal comprehension scores were not included in the analysis because of ceiling effects.
With regard to language of administration, although some statistically significant differences in test performances were found between individuals who were assessed in Spanish, and those who were examined in Catalan (see Table 14), most of general cut-offs obtained were similar in both languages (see Table 15). However, it was not possible to obtain cut-offs taking at the same time the age, education and language because that would be supposed an excessive reduction of the sample. Thus, after combining age by education by diagnostic and by language, there were 9 cells (the 37.5%) with, for example, less than 30 subjects.

Discussion
The present study reports the relevant diagnostic cut-off scores for a brief neuropsychological battery, the NBACE, for use in identifying subjects with cognitive impairment, especially dementia, in adults older than 44 years old. To our knowledge, this is the first study reporting sensitivity, specificity and AUC parameters of a neuropsychological battery for diagnosis of dementia in our country.
The AUC results demonstrated that most of NBACE cut-off scores were useful for detecting MDS in a sample of Spanish-Catalan speaking subjects, except for Repetition of sentences, Verbal Comprehension and Ideomotor praxis, which had ceiling effects in most of cases. Although Digit Span Forward subtest of WAIS-III showed a lower AUC value, the general cut-offs had good sensitivity and specificity values. The measures of verbal learning and long-term memory (including its recognition task) and Semantic Verbal Fluency were the most useful for detecting the presence of cognitive impairment or dementia. This is consistent with previous studies [47,48,49], and is probably due to the fact that most of MDS patients suffered AD. As a limitation of the study, it has to be mentioned that participants were assessed in their mother tongue, but many of them spoke as well Catalan as Spanish. Then, it is uncertain about the percentage of subjects with balanced or unbalanced bilingualism (that is, with the ability to express high-proficiency equilibrate in both languages or not, respectively).
However, understanding the relevance of comparing performances and cut-offs between Spanish and Catalan groups, they have been reported. Non-verbal cognitive functions, such as visuospatial and visuoconstructive abilities showed statistically significant differences between groups, whereas for verbal cognitive functions, only visual naming showed differences between Catalan and Spanish speakers. Remarkably, in healthy and mild dementia groups, the Catalan group obtained better performances on visual naming than the Spanish one. Although further studies are needed to improve the knowledge about NBACE performances between bilingual and non-bilingual subjects, this finding is according with previous studies, reinforcing that bilingualism improves performance on visual naming in healthy subjects [50], but also in patients with dementia [51].
The results of the present study fit in with results from other previous studies, such as those from the Spanish Multicenter Normative Studies (NEURONORMA Project) [10][11][12][13], in an effort to provide norms for the neuropsychological assessment of cognitive functions in Spain. Moreover, as reported previously [19], the NBACE comprises an exhaustive clinically useful neuropsychological test battery that covers all the cognitive domains affected by neurodegenerative disorders. It can be administered in approximately 45 minutes, making it feasible to use in a clinical setting. The NBACE has been shown to be a useful tool able to detect cognitive impairment, especially dementia, in older than 44 years Spanish-speaking persons.
Data reported here are the next step of a previous study [19] in setting the relevant cut-off scores for impaired function, for different levels of age and education, and decision making algorithms for use in identifying adult individuals with cognitive impairment. Fundació ACE enrolls more than 1000 new patients every year, all of whom are assessed by the NBACE.
In the present study, the neuropsychological assessment was satisfied in a Diagnostic Unit with specific demands and needs. This is an intrinsic limitation for future replication processes when other disorders or severities are considered. To validate these cutoff scores in other clinical samples, replication studies would be useful. As a next step, further studies are needed to assess the usefulness of NBACE tests battery in discriminating among different types of dementia.