The Arrows and Colors Cognitive Test (ACCT): A new verbal-motor free cognitive measure for executive functions in ALS

Background and objective The presence of executive deficits in patients with Amyotrophic Lateral Sclerosis is well established, even if standardized measures are difficult to obtain due to progressive physical disability of the patients. We present clinical data concerning a newly developed measure of cognitive flexibility, administered by means of Eye-Tracking (ET) technology in order to bypass verbal-motor limitations. Methods 21 ALS patients and 21 age-and education-matched healthy subjects participated in an ET-based cognitive assessment, including a newly developed test of cognitive flexibility (Arrows and Colors Cognitive Test–ACCT) and other oculomotor-driven measures of cognitive functions. A standard screening of frontal and working memory abilities and global cognitive efficiency was administered to all subjects, in addition to a psychological self-rated assessment. For ALS patients, a clinical examination was also performed. Results ACCT successfully discriminated between patients and healthy controls, mainly concerning execution times obtained at different subtests. A qualitative analysis performed on error distributions in patients highlighted a lower prevalence of perseverative errors, with respect to other type of errors. Correlations between ACCT and other ET-based frontal-executive measures were significant and involved different frontal sub-domains. Limited correlations were observed between ACCT and standard ‘paper and pencil’ cognitive tests. Conclusions The newly developed ET-based measure of cognitive flexibility could be a useful tool to detect slight frontal impairments in non-demented ALS patients by bypassing verbal-motor limitations through the oculomotor-driven administration. The findings reported in the present study represent the first contribution towards the development of a full verbal-motor free executive test for ALS patients.

Introduction modality, allowing patients to respond verbally or manually [20][21][22]. However, even the presence of minimal motor functions could be prevented in the advanced stages of the disease.
Eye Tracking (ET), a well-known and consistently validated Alternative and Augmentative Communication system, has been used with the aim of administering neuropsychological tests in a motor-verbal free manner [23][24][25][26]. We recently presented preliminary results showing an extensive neuropsychological battery developed for ET control, covering language, attentional and executive functions, and social cognition domains [27][28][29]. The current study describes clinical data on a newly developed measure of cognitive flexibility, the Arrows and Colors Cognitive Test (ACCT), administered by means of ET technology. Relationships between other validated standard measures and with the recently proposed oculomotor-driven neuropsychological battery are discussed, together with sensitivity in discriminating between ALS patients and healthy controls.

Participants
Twenty-one ALS patients were recruited at the inpatient-outpatient ALS Center at the Department of Neurology of the IRCCS Istituto Auxologico Italiano, Milan. The diagnosis of definite ALS was made by an expert consultant neurologist, according to El Escorial Criteria [30]. Patients were excluded if they were in terminal stages of disease or had major medical, neurological, psychiatric history, and/or cardiovascular comorbidities. None of the patients had major visual impairments or oculomotor dysfunction. Twenty-one healthy subjects, matched for age and educational levels, were also recruited. All participants were native Italian speakers.
For ALS patients, clinical and laboratory assessments included evaluation of disability, using the revised ALS Functional Rating Scale (ALSFRS/R) [31], and of respiratory function with spirometry.
All participants completed the designed cognitive and psychological protocols and the ETbased assessment, as described below.
The study protocol was reviewed and approved by the Ethics Committee of IRCCS Istituto Auxologico Italiano (ethics committee code: 2011_04_12_16) and all eligible subjects received verbal and written information about the study. All participants signed an informed consent, according to the Declaration of Helsinki. The individual in this manuscript (S1 and S2 Videos) has given written informed consent (as outlined in PLOS consent form) to publish these case details.

Cognitive and psychological assessment
ACCT: ET-based assessment of cognitive flexibility. ACCT is composed by four rows (ACCT-1, ACCT-2, ACCT-3, ACCT-4), made of 12 items each, where participants are required to select the appropriate arrow according to a written instruction. The patients were allowed to practice each trial twice before the start of data collection. Two types of instructions were presented. In the first (ACCT-1, see S1 Video; the individual in this manuscript has given written informed consent, as outlined in PLOS consent form, to publish these case details) and third (ACCT-3) trials: "Note the arrow in the upper center. In the following exercise, you will be asked to choose between the arrows below, the one with the different color, but the same direction as that of the target arrow". In the second (ACCT-2) and fourth (ACCT-4, see S2 Video; the individual in this manuscript has given written informed consent, as outlined in PLOS consent form, to publish these case details) trials: "Note the arrow in the upper center. In the following exercise, you will be asked to choose, between the arrows below, the one with the same color, but a different direction from that of the target arrow". Upper arrows are equally distributed between four directions (3 arrows for each direction). Target arrows are equally distributed between the right, left and central positions (4 for each position in 3-arrow trials, 3 for each position in 4-arrow trials). For each task, two or three distracters were present for ACCT-1 and ACCT-2 and for ACCT-3 and ACCT-4 respectively; distracters were represented by items following a part of the instruction (i.e. color or direction), and by a perseverative response over the previous subtest (see Fig 1). The mean latency, mean latency standard deviation, and the total number of correct responses were recorded for each subtest. Moreover, partial errors (i.e. following Oculomotor-based executive assessment in ALS PLOS ONE | https://doi.org/10.1371/journal.pone.0200953 August 9, 2018 a part of the instructions) and perseverative errors were computed for each subtest, in order to obtain additional qualitative data about subjects' performances.
Other standard and ET-based cognitive and psychological tests. A neuropsychological and psychological protocols employed in previous studies were adopted, including both standard and ET-based cognitive measures. In particular, standard cognitive measures included two brief batteries, for dysexecutive syndrome, FAB [32], and global cognitive functioning, MoCA [33], respectively, and a working memory-WM subtest, Digit Sequencing Task of the Brief Assessment of Cognition in Schizophrenia (BACS) [34].
In addition, a battery of neuropsychological tests, previously adapted to oculomotor control by means of ET technology, was employed for evaluating language (Token Test), attentional abilities (d2 Test), executive functions (Raven's Colored Progressive Matrices-RCPM, Modified Card Sorting Test-MCST, Iowa Gambling task-IGT) and social cognition (Reading the Mind in the eyes test-RME) [28,29].
In addition, participants completed a screening for symptoms of depression and anxiety, using both the Beck Depression Inventory (BDI) and the State-Trait Anxiety Inventory-Y (STAI-Y). BDI was used to assesses both cognitive-affective (BDI-CA) and somatic-performance (BDI-SP) symptoms of depression [35], and STAI-Y evaluated both state anxiety level (STAI-Y1) and trait anxiety (STAI-Y2) components [36]. For ALS patients, a behavioral assessment was also performed using the Frontal Behavioral Inventory (FBI) [37].

Eye-Tracking
An EyeLink 1000 infrared tracker was used to record participants' eye movements throughout all the experimental conditions. Head position was stabilized using a chin rest, at a distance from the screen of around 70 cm. A nine-points calibration covering the totality of the visual screen and a drift correction were carried out before each test, in order to ensure an accurate eye position recording. In order to select an item on the computer screen, participants were required to fixate on the item for a minimum of 80% over a period of 1,500 milliseconds. Eye movement data consisted of moment-to-moment measures of eyes' displacements along the vertical and horizontal axes (in millimeters) within the spatial working area of the monitor screen (resolution 1024 x 768 pixels). Pupil dilation and gazes were acquired, based on pupil position and corneal reflection on the frontal surface of participants' eyes (caused by an infrared light source), at 250 Hz by means of an EyeLink 1000 system using a new software suite, named the eBrain Test Engine (ETE, http:// ebrainengine.codeplex.com/). Acquired signals were also analyzed using the following off-line tools: Experiment Builder (SR Research, Ottawa, Canada), Excel (Microsoft, Redmond, USA) and custom programs written in MATLAB 7.2 (The Mathworks, Inc.; Natick, MA). A training phase was performed, as described in previous works [28,29].

Procedure
The overall experimental procedure was performed along two sessions within a one week period. In the first session, standard cognitive tests and psychological measures were administered. In addition, the subjects participated in a training phase for the ET evaluation. During the second session, participants underwent the ET-based neuropsychological assessment. The administration order of each ET neuropsychological test was randomized among subjects, in order to avoid practice and fatigue effects.

Statistical analysis
Descriptive statistics (mean ± standard deviations for continuous variables and frequencies for discrete variables) were used to describe the main characteristics of the recruited samples and performances of the ET and standard cognitive/psychological assessments. Distribution of the variables in terms of proximity to normal curve and the homogeneity of variances were detected by the Shapiro-Wilk test and Bartlett test, respectively. To compare the mean scores between groups, a two-sided t-test with pooled estimates of the sample variance or the Welch approximation were employed when the continuous variables were normally distributed, homoscedastic or heteroscedastic, respectively. Otherwise, a two-sided signed-rank test was performed. Finally, to assess the degree of association between scores in the ALS patients, the Spearman correlation coefficient was adopted. P-values were adjusted with for a false discovery rate.
To detect the sensitivity and the specificity of ET test, the area under the receiver operating characteristic (ROC) curve (AUC) was evaluated; cut off values were identified according to the Youden's J statistic method [38]. An α level of 0.05 was used for all hypothesis tests. All data analyses were performed using R Core Team (2014), Vienna, Austria.

ACCT results: Between groups differences
The performances of patients and controls were different when assessed through ACCT, with significantly higher values of mean latency observed for patients at ACCT-1, ACCT-2, and ACCT-4 subtests. Significantly higher values of the mean latency standard deviation were observed for patients for the ACCT-1 and ACCT-4 assessments, suggesting a larger variability of performances in this group compared to the control group. A tendency towards a significant difference in number of correct responses was observed for ACCT-4, with lower scores for patients (Table 1). Significantly higher values of mean latency for only correct responses were observed in ALS patients' group, for ACCT-1, ACCT-2, and ACCT-4 subtests (S1 Table).
The ROC curve analysis supported the sensitivity of ACCT-1, ACCT-2, ACCT-4 mean latency and mean latency standard deviation variables (AUC values >0.70), according to the determined cut-off scores. AUC values between 0.6 and 0.7 were found for ACCT-3 number of correct responses and for ACCT-3 mean latency.
A qualitative analysis on patients' performances of the types of errors committed highlighted an increasing number of perseverative errors during the ACCT-3 to ACCT-4 subtests. However, perseverative errors were globally less present than partial ones, with a prevalence of partial (color) errors in ACCT-1, ACCT-3 and ACCT-4 subtests. For ACCT-2, a higher proportion of partial (direction) errors, with respect to partial (color) ones, was observed (Fig 2).

ACCT results: Correlation with other standard and ET-based cognitive tests
When considering the ALS group (Table 2), significant correlations were observed between MoCA and ACCT-1 mean latency and number of correct responses; between FAB and ACCT-3 number of correct responses. Correlations between ACCT and WM subtest only concerned mean latency standard deviation of ACCT-2, even if a tendency toward a significant correlation (p-value = .06) was observed for the mean latency standard deviation of ACCT-3, and number of correct responses for ACCT-2. For controls (Table 2), significant correlations were observed between MoCA and the number of correct responses of ACCT-3, with a mean latency trend close to significant (p-value = .06) for ACCT-3; between WM subtest and the mean latency standard deviation of ACCT-2, with a trend toward a significant correlation (pvalue = .06) for mean latency correlation of ACCT-2. Conversely, the FAB did not correlated with any of the ACCT variables in controls' groups.
When correlations between ACCT and other ET-based neuropsychological tests were considered, the ALS group showed associations between mean latency and mean latency standard deviation of the ACCT subtests and frontal-executive measures (d2, MCST, RCPM, IGT), together with RME as a measure of social cognition (Table 3 and S2 Table). Response accuracy of the ACCT-2 subtest correlated with RME (number of correct responses) (S2 Table). For controls, correlations were less extended and mainly regarded mean latency and mean latency standard deviation of ACCT-2 and ACCT-3 subtests that correlated with frontal-executive measures and control task at the RME (Table 3 and S3 Table). In this group, response accuracy for the ACCT-1 subtest correlated with d2, MCST and RCPM measures; some trends were also observed between response accuracy for the ACCT-3 subtest and the RCPM and RME control task (S3 Table).

ACCT results: Correlation with clinical and psychological parameters
No significant correlations were observed between ACCT variables (mean latency and number of correct responses), and either the disease onset or ALSFRS/R in the ALS patients' group. With regard to psychological components in the patients' group, a negative correlation was observed between state anxiety level (STAI-Y1) and mean latency standard deviation for the ACCT-2 (rho = -.47; p-value = .031) and ACCT-3 (rho = -.59; p-value = .006) subtasks. Trait anxiety (STAI-Y2) was positively correlated in this sample with number of correct responses at the ACCT-3 subtask (rho = .45; p-value = .043).
No significant correlations were observed between ACCT subtest variables (mean latency, mean latency standard deviation and number of correct responses) and patients' behavioral examination (FBI).

Discussion
The ACCT we presented, highlighted the decreased performances of time-related measures of ALS patients, when compared to controls. Less consistent results were observed with accuracy-related measures, i.e. number of correct responses, with a trend towards a significant difference observed across trials, from ACCT-1 to ACCT-4. ACCT-3 seems slightly more difficult than the other subtests, according to the observed mean values and the lack of differences between patients and controls performances. Maybe, the increase of arrows numerosity from ACCT-2 to ACCT-3 and the progressive instructions alternation involve a more significant working memory load, as also suggested by the appearance of perseverative errors.
Overall, our findings suggest the presence of slight cognitive changes concerning cognitive flexibility and set-shifting abilities in our non-demented ALS sample that may overtly affect execution times and, to a lesser extent, performance accuracy. Recent findings have highlighted functional reorganization in ALS patients during cognitive tasks as an early adaptive process to neuronal cell loss and time-related aspects could represent the initial clinical signs of cognitive change in our ALS population [39]. Longitudinal evaluation of patients' performances using the ACCT test and other cognitive tasks could identify changes in performances across disease progression and better characterize cognitive changes. Furthermore, it is worth noting the few patients with ALSci in our sample (12.5%). Our small sample size could explain the limited presence of cognitive involvement detected by the ACCT.  In contrast with our results, in a recent study, Pettit and colleagues concluded that processing speed, when isolated from motor functions and high-order cognitive processes, is preserved in ALS patients, and, therefore, does not contribute to the frequently observed cognitive changes in such a population [9]. A possible explanation of these controversial findings could be that, in our study, the use of an oculomotor-driven version of the cognitive tests could affect performance speed, due to the possible presence of ocular motor disorders as a marker of sub-clinical frontal lobe dysfunction observed in ALS patients [40][41][42]. However, the clinical examination that preceded the administration of the experimental protocol excluded the presence of significant oculomotor dysfunctions. Moreover, the employed paradigm minimized the oculomotor component necessary to provide responses to the presented items. Finally, no correlation between patients' physical function decline (ALSFRS/R) and mean latency at the ACCT subtests was observed. Similarly, disease onset did not correlate with time-related variables provided by the ACCT test. Overall, such results suggest that motor limitation severity and progression did not influence the speed of processing in our sample, and that such time-related aspects are likely related to an underlying cognitive involvement.

ALS patients
Healthy controls Correlations between ACCT and other ET-based measures of frontal-executive abilities were extended and congruent, and mainly involved the mean latency variables of ACCT. Some relationships have also been observed between ACCT and a measure of social cognition, RME, suggesting a role for set-shifting and frontal-executive abilities in such function, as reported previously [4].

ACCT-3 Mean latency
Otherwise, ACCT demonstrated limited correlation with other standard cognitive tests, such as the MoCA and a WM measure (Digit Sequencing Task), and very poor correlation with a widely used battery that assesses frontal abilities (FAB). Discrepancy between the FAB and ACCT tests could be explained by the poor sensitivity of the 'paper and pencil' frontal battery in non-demented ALS patients, as suggested by recent literature [43] and confirmed by our study. Moreover, it is also likely that FAB and ACCT assessments target different components of executive function. With regard to WM subtest, although several studies have proposed a relationship between frontal-executive and WM abilities [44][45][46] and mild dysfunction of WM abilities are consistent with previous, multidimensional reports [5,47], we observed limited correlation in our study between ACCT performances and WM subtests. Further refinements of the developed test, aimed at increasing the level of difficulty and the sensitivity of the different subtests, as well as comparison with other WM assessment tools, could provide more consistent data.
In addition to time and accuracy-related variables, the developed ACCT test allows the collection of more qualitative data about frontal-executive involvement than other available tests, according to the categorization of errors presented. The preliminary findings presented here suggest a reduced presence of perseverative errors, with respect to other type of errors, in our sample of non-demented ALS patients. Such results are in accordance with a previous study using a similar population where impaired inhibition was not a prominent feature [10]. Moreover, the use of a sophisticated and computerized tool, such as ET, could enable the collection of findings that would provide insight into the nature of errors that patients make when scanning items (i.e. perseverative, due to impulsivity or disinhibition). Further refinements of this protocol, including detection of oculomotor performances, and recruiting patients with more heterogeneous and severe cognitive impairments, could improve the test sensitivity and provide more subtle indications about cognitive performances.

Conclusions
In ALS, frontal abilities, and particularly cognitive flexibility, are known to be impaired due to limitations in executive control in such patients. However, observations supporting such changes have been often collected by means of tests that do not accommodate for verbal and motor disabilities. The detection of cognitive impairments during the course of the disease is not fully and reliably accomplished by means of 'paper and pencil' tests, because the motor and verbal limitations are severe in moderate-severe stages of ALS. Such issues may at least partially account for the inconsistency of findings obtained by previous studies about specific components of cognitive involvement in the disease, including subcomponents of frontalexecutive abilities.
We presented preliminary results about an extensive neuropsychological battery developed for oculo-motor control by means of ET, covering language, attentional and executive functions, and social cognition domains. The current study provides clinical data of a newly developed measure, the Arrows and Colors Cognitive Test (ACCT), administered by means of ET technology. The newly developed test allows a specific assessment of cognitive flexibility. Such measures, when integrated to other complementary tasks, could provide a more specific evaluation of different frontal changes involved in the course of ALS.
Our results propose the use of ACCT as a tool to assess cognition in patients with verbalmotor disabilities, such as ALS, when standard measures are not fully administrable. Further efforts will be aimed at investigating the feasibility of the developed system, together with validity and usability components, in larger populations of ALS patients. In particular, the inclusion of ALS patients presenting overt cognitive or behavioral impairments, or FTD, will be useful for improving ACCT clinical sensitivity. Finally, future research will be aimed at enhancing the ACCT number of items and level of difficulty in order to improve both its sensitivity and accuracy.
Despite these limitations, the findings reported in the present study represent the first contribution towards the development of a full verbal-motor free executive test for ALS patients. Accommodating or compensating for verbal-motor component is a crucial issue for longitudinal assessment of cognition in ALS. Adequately assessing such cognitive components could enhance clinical observations and better describe ALS patients' phenotypes along the course of the disease, avoiding biases present in clinical and research settings.
Supporting information S1 Table. Performance on ACCT subtests in ALS patients and healthy subjects, with concern to correct responses. Data are expressed as Means (SD). (DOCX)

S2 Table. Correlations between ACCT subtests and other ET-based tests in patients' group.
Bold numbers indicate statistical significance with p < 0.05. (DOC)