Visual Search as a Tool for a Quick and Reliable Assessment of Cognitive Functions in Patients with Multiple Sclerosis

Background Despite the high frequency of cognitive impairment in multiple sclerosis, its assessment has not gained entrance into clinical routine yet, due to lack of time-saving and suitable tests for patients with multiple sclerosis. Objective The aim of the study was to compare the paradigm of visual search with neuropsychological standard tests, in order to identify the test that discriminates best between patients with multiple sclerosis and healthy individuals concerning cognitive functions, without being susceptible to practice effects. Methods Patients with relapsing remitting multiple sclerosis (n = 38) and age-and gender-matched healthy individuals (n = 40) were tested with common neuropsychological tests and a computer-based visual search task, whereby a target stimulus has to be detected amongst distracting stimuli on a touch screen. Twenty-eight of the healthy individuals were re-tested in order to determine potential practice effects. Results Mean reaction time reflecting visual attention and movement time indicating motor execution in the visual search task discriminated best between healthy individuals and patients with multiple sclerosis, without practice effects. Conclusions Visual search is a promising instrument for the assessment of cognitive functions and potentially cognitive changes in patients with multiple sclerosis thanks to its good discriminatory power and insusceptibility to practice effects.


Introduction
With a prevalence of 40 to 65% [1,2], cognitive impairment is one of the most common symptoms of multiple sclerosis (MS) that may already arise at an early stage of the disease [3]. Typically affected domains are memory, attention, information processing and executive functions, whereas language and simple attention span are scarcely impaired (for review: [4][5][6]. Notably, cognitive deficits negatively affect patients' quality of life [7], ability to work [8] and rehabilitation outcome [9]. Despite those debilitating consequences of cognitive impairment, it is not routinely assessed during clinical examination or trials. One reason for this shortcoming is that most available cognitive tests have been designed for other patient groups (i.e. stroke) and thus are not sensitive enough to assess cognitive functions in MS, where, particularly during an early stage, only mild deficits predominate. Moreover, to cover the variety of potentially affected functions in MS the conduction of comprehensive test batteries is required, making assessment time-consuming and impracticable. Short screening tests (i.e. Paced Auditory Serial Addition Test; PASAT) are susceptible to measurement errors, assess only few facets of cognitive functioning and are not suitable to assess changes in cognitive functions due to considerable practice effects [10]. Self-reports are just as little suited for the assessment of cognitive deficits, since they strongly correlate with depressive symptoms [4] and diverge with objective measures of cognitive impairment [6].
The importance of the assessment of cognitive impairment in MS patients on the one hand and the lack of suitable instruments for this purpose on the other hand, emphasize the need to identify a suitable measure of cognitive deficits in MS patients circumventing the problems of currently available tests. Such a test could be the visual search task. This paradigm was introduced by Treisman and Gelade [11] and is one of the best understood tasks in the field of cognitive neurosciences. In this task, a target stimulus, presented on a computer screen, has to be detected amongst similar distracting stimuli via button press (see Figure 1).
Depending on the displayed array of stimuli visual attention or purely perceptual processes (figure ground segregation) are required [11]. This allows the separate assessment of sensory and attention-related deficits. Additionally, position priming can be induced by repeating the target location leading to faster responses, which is an indicator of implicit perceptual memory [12]. Apart from priming, practice effects are typically not observed [13], suggesting that the visual search task can be reliably used to assess changes in cognitive functions via repeated testing. To broaden the diagnostic information of the visual search task, the response key can be replaced by a touch screen, requiring the subject to directly touch the target on the screen [14]. In this way, indicators of visuomotor functions can be assessed such as movement time or accuracy of pointing. Moreover, using a pointing response instead of the more conventionally employed button-press response offers one further advantage. Conventionally the participant has to decide whether a target item is present or not and press the corresponding button to indicate their decision. However, this means that a substantial number of trials without a target item (catch trials) have to be presented. These trials are frequently not used for the final analysis. When a pointing response is used, the accuracy of the pointing response is proof that the target item has been detected. Catch trials are thus superfluous and thus more data can be collected in less time.
To sum up, the visual search task seems to be an ideal tool for the assessment of cognitive impairment in MS: It assesses different facets of cognitive functions and motor behaviour, shows no practice effects and thus in principle could be able to assess disease-or medication-related changes in cognitive functions of MS patients.
In the present study we examined MS patients and healthy individuals both with our modified visual search task and with a set of standard neuropsychological tests. Our aim was to determine which cognitive assessment procedure would provide the most reliable discrimination between individuals with and without MS.

Ethics Statement
The protocol was conducted in accordance with the Declaration of Helsinki II and was approved by the ethics committee of the Friedrich-Alexander University Erlangen-Nuremberg. All participants gave their written informed consent prior to the investigation.

Participants
Thirty-eight outpatients (11 males, 27 females) with relapsing remitting MS according to the 2005 revised McDonald criteria [15] with a mean age of 36 years (standard deviation SD: 10.57; range: 21-60) were consecutively recruited. All patients were eligible for escalation therapy, meaning that they had experienced a relapse during medication and had shown signs of relapse as demonstrated by MRI in the preceding year, but had not experienced a relapse within 30 days prior to the examination, which was an exclusion criteria. Furthermore, patients were excluded in case of drug or alcohol abuse, diagnosed dementia or depression, an Expanded Disability Status Scale (EDSS) [16] score above 5.5, high-dose steroid therapy, or whenever another immune disease requiring immune suppression was diagnosed.

Neuropsychological standard tests
A selection of seven neuropsychological tests was used as our baseline measurement. We selected tests which had been used before in MS research or formed part of a cognitive test battery for MS patients and focussed on those cognitive deficits which have been most commonly found in MS patients, namely deficits of attention and memory (see Table 1).

Visual search task
In this task, participants had to search a target amongst a varying number of distractors. Three different versions of visual search tasks were conducted (see Figure 1). All tasks were programmed in E-prime 2 (Psychology Software Tools, Inc., Pittsburgh, PA). All stimuli were presented against a black background on a 17" computer monitor covered by an add-on touch screen (KTMT1700 USB; Magic touch, Keytec Inc., Garland, Texas) and driven by a Pentium M 750 notebook. Table 2 shows the different parameters of the search tasks, and Figure 2 illustrates the sequence of each trial.
Reaction (search) time (until button release), movement time (time between button release and target touch) and pointing error (distance between touch and target location in mm) were computed (see Table 3). Furthermore, the percentage difference between the reaction time for the first and third presentation of priming trials was computed as an indicator of implicit location learning. The assumption being that when participants remember the location of the previous trials, they should take less time to detect the target, thus leading to shorter reaction time for the third repetition and thus a bigger difference between reaction times for the first and third presentation of the same target location.

Procedure
All participants were tested separately in a quiet room. The tests were conducted in the following order: 1. SPART 2. Go/Nogo 3. Divided attention 4. SPART delayed recall 5. Digit span 6. Spatial span 7. Logical Memory I 8. PASAT-3'' 9. Visual search Twenty-eight participants (seven males, 21 females) of the healthy control group with a mean age of 35.6 years (SD: 11.49) were re-tested with the same tasks three to six months later (mean: 3.56; SD: 0.79), in order to check for potential practice effects. When parallel versions were available, the parallel test versions were used for the second assessment (SPART, pattern 2; PASAT-3'', version B). In the case of Logical Memory I, we created a parallel version by using the first story for the first assessment and the second story for the second test session.

Statistics
Mixed 2x3 ANOVAs with the between factor "group" (MS, healthy) and the within factor "visual search task" (single feature search -colour, feature conjunction search, difficult feature search -orientation) for mean reaction time and movement time were conducted. Independent sample t tests were computed to assess group differences. Paired-samples t tests were performed to determine potential practice effects. Furthermore, a receiver operating characteristic (ROC) analysis was conducted, including the tests for which significant group differences were detected. Spearman's correlation was used to determine the relationship between the measures of visual search and EDSS score, visual function score within the EDSS and time since diagnosis in MS patients respectively. The alpha level was set at p=.05 for all analyses, which were computed with IBM SPSS Statistics 19.

Neuropsychological standard tests
There were significant differences between MS patients and healthy individuals in the PASAT-3'', Digit span backward, Logical Memory I, and Divided attention test, with MS patients generally showing inferior performance (see Table 3). No significant group differences were found for the remaining tests. As the discriminatory power between the three tasks were comparable (see ROC analysis) despite the interaction between group and visual search task, the following analyses were conducted using the combined data of the three visual search tasks.

Visual search task
The same was done for mean movement time. For movement time no significant group x visual search task interaction (F(2,152) = 0.261; p=.77) was found. MS patients also showed a significantly prolonged movement time Visuospatial learning and delayed recall In our modified paper-and-pencil version the original pattern was presented on a sheet of paper (ten circles in a 6x6 grid pattern) for 10 seconds. The participants' task was to replicate the spatial pattern as precisely as possible. They did this by drawing circles into the corresponding cells of an empty grid pattern on another sheet of paper. This procedure was repeated twice. After 15 minute delay (delayed condition) participants were asked again to reproduce the spatial pattern from memory, but this time the spatial pattern had not been shown again. In case more than 10 target circles were produced, the number of excess circles was subtracted from the number of correctly assigned circles. For the immediate condition all three trials were combined yielding a maximal score of 30; for the delayed condition a maximum score of 10 could be achieved Paced Auditory Serial Addition Test 3'' (PASAT [30,31]) Auditory information processing speed and selective attention [30] Sixty-one numbers, orally presented by tape every 3 seconds (PASAT-3ˡˡ) have to be added such that the just heard number has to be summed up to the previously presented number. The total of correctly added pairs of numbers is counted (maximal 60) Wechsler Memory Scale Revised Edition (WMS-R [32]); German edition [33] Digit span forward/backward Verbal short-term memory Sequences of digits increasing in number are orally presented and the subject has to either repeat it in the original (forward) or the reversed order (backward). The number of correctly reproduced sequences is scored, separately for the forward and backward trials Spatial span backward/forward Visuospatial shortterm memory Eight blocks on a board are presented. The experimenter taps a certain sequence of blocks and the subject has to immediately reproduce the sequence in forward or backward order. The number of correctly reproduced sequences is scored, separately for the forward and backward trials Logical memory I (modified) Verbal long-term memory For this test we used a simplified version, i.e. only one of the two stories was used and one point was given whenever the item was recalled with its precise wording, half a point was awarded, when the correct meaning was recalled, but the wording differed. To ensure that our scores were comparable to those obtained with the original version of this test, we multiplied the final score by a factor of 2. Thus the maximum score in our version was identical to that of the original version, namely: 50 Testbatterie Selective attention A series of five squares filled with different patterns are presented in random order on a computer screen. Two of them are critical target stimuli that require a button press as quickly as possible, whereas no response is needed to the other three stimuli. Reaction time and the number of errors are scored Divided attention (version I/auditive-visual) Divided attention A quadratic array of 4x4 dots is displayed. Between six and eight Xs appear on the locations of the different dots. The participant is required to press a button as quickly as possible, whenever four of these Xs constitute a square. Simultaneously, a high-pitched and a deep tone are presented in alternate order. Whenever the same tone appears in succession, the participant has to press a button as quickly as possible.  Table 4 for further details). There were no significant differences between groups regarding priming effects and spatial pointing error. No significant correlations were detected between EDSS score and reaction time or movement time, or time since diagnosis and reaction or movement time (all p's>.05). Likewise, no significant correlations were found between the measures of visual search and the visual function score (all p's>.05). There were no significant differences between patients with (n = 14) and patients without (n = 24) movement deficits of the right  Figure 2. The sequence of a trial using the example of the single feature search. At the beginning, a central fixation cross was displayed along with the instruction to hold down the start button. Participants had to press a button on a Serial Response Box (Psychology Software Tools, Inc., Pittsburgh, PA). At the moment of button press, the stimuli appeared on the screen. The participants were instructed to visually search the target, and release the button only after they had detected the target. Next they should touch the target on the screen. The trial was completed (and the next trial initiated) when the touch screen detected a touch or when more than 3000 milliseconds (ms) had elapsed since button release. Trials, that were aborted because no touch was registered within the 3000 ms time window, were excluded from the analysis.  Figure 3 illustrates the ROC-curves and Table 5 shows the respective Areas Under the Curve (AUC). Mean reaction time and movement time in the different visual search tasks, as well as for the combined data of all three tasks, turned out to discriminate best between healthy individuals and MS patients, as indicated by the highest AUC values.

Practice effects
There were significant differences between the first and second measurement in the re-tested healthy individuals for the PASAT-3'' and Digit span forward with better performance during the second measurement (see Table 6). Marginally significant differences were found for SPART and divided attention. The performance in the remaining tests did not differ significantly between the two testing sessions.

Discussion
We compared a selection of commonly used cognitive tests with the paradigm of visual search regarding their ability to discriminate between healthy individuals and MS patients and regarding their potential practice effects. Reaction time and movement time in the visual search task turned out to discriminate best between the two groups, without showing any practice effects, confirming the results of Ball et al. [13]. This   suggests that the visual search task is a suitable method to assess cognitive impairment in MS patients and furthermore potentially disease-or medication-induced changes in MS patients due to robustness against practice effects. Inferior mean performance in visual-attentional functions were observed in the visual search task for MS patients as indicated by longer reaction times for this group compared to the healthy individuals. However, no group differences  regarding priming effects were found, meaning that implicit learning was unimpaired in our MS patients, as previously reported [1]. This is in line with findings in Alzheimer's patients who also demonstrate intact perceptual priming [17] suggesting that implicit memory is a surprisingly robust skill that remains intact even when many other cognitive functions decline. The lack of group differences concerning reaction time between patients with and without movement deficits of the right arm, as well as the lack of correlation between reaction time and visual deficits, suggests that this outcome measure is relatively robust against mild motor and visual defects.
Findings from our battery of standard neuropsychological tests mirrored those from previous studies: MS patients showed significantly lower performance in tests assessing verbal long-term memory [18], divided attention [3], information processing speed [2], as well as verbal short-term memory/ working memory (digit span backward [19,20]), but not in the simpler version of this task (forward [2]). However, for all of those standard tests the power to discriminate between healthy and MS participants was inferior to that of the visual search task. Interestingly, even the most commonly used test, the PASAT, showed considerable lower discriminatory ability and was moreover prone to practice effects, providing further ammunition for those who are critical of the PASAT's usefulness as a reliable measure of cognitive function [10].
Given the increasing recognition of cognitive impairment in MS, different strategies have been proposed for its assessment, such as test batteries like the BRB-N [21] or the Minimal Assessment of Cognitive Function in Multiple Sclerosis (MACFIMS) [22], patient-or family member-reported scales [23,24] and different screenings such as single tests [25], a collection of tests [26] or internet-based cognitive testing [27]. However, whereas test batteries are too time-consuming, selfor family member-reported scales are not reliable [4,6], and many screening tests assess only one or few, mostly speechdependent aspects of cognition and are often susceptible to measurement errors and practice effects, thus being inappropriate for the assessment of changes in cognitive functions [10,28].
The administration duration of the visual search task was between 15 and 20 minutes. However, as there were no differences regarding the performance during the first three and the second three blocks, the task duration could be shortened to seven to 10 minutes, thus being shorter than most of the available screening procedures. In fact given that all three search tasks discriminated equally well between participants with and without MS, it would be feasible to restrict the examination to just one of the three search tasks, thereby reducing the examination time to approximately 5 minutes.
Despite its brief administration duration, the visual search task assesses many different facets of cognitive functions, especially those frequently neglected by conventional declarative tests, such as visual exploration and visual memory.
Correspondingly, Feinstein et al. [29] recently found that the phenomenon of inattentional blindness, whereby healthy individuals fail to notice a perfectly visible person/object when they are concentrating on another task, is observed less in MS patients. This is taken as a sign of increased distractibility in MS patients. Interestingly, this pervasive deficit is not picked up by the MACFIMS [22] battery: MS patients with abnormal performance in the inattentional blindness task showed normal performance in the MACFIMS tests. The authors conclude that this could explain why some MS patients have difficulties in daily life situations requiring the filtering of distractors or multitasking without showing deficits in test batteries [29]. The visual search task assesses distractibility quite explicitly since successful performance requires that participants focus on a given target feature while filtering out irrelevant but often salient distracting features. This aspect of the visual search paradigm may in part account for its superior power to discriminate between participants with and without MS.
The visual search task offers other advantages which so far have not been mentioned. It is computer-based, therefore can be carried out by the patients without supervision and can also be automatically analyzed, thus yielding objective test results with minimal effort.
Since the mean disease duration of our sample of MS patients was over seven years, we do not know whether the same results can be expected for a sample of early-stage MS patients. We should also note another limitation of our study. Our procedure to identify sensitive cognitive tests was based on a comparison between healthy participants and MS patients. Tests which discriminate reliably between MS patients and healthy adults will help in the diagnosis of MS patients but may not provide the best measures to examine the impact of clinical drugs on disease progression. To find out whether our visual search tasks provide the sensitivity to detect spontaneous or therapy-induced changes over time a longitudinal study will be needed.
In conclusion, visual search seems to be a good tool for the assessment of cognitive impairments in MS due to its good discriminatory power, easy and objective computer-based administration, high accuracy of measurement due to the great number of trials, and assessment of a wide range of cognitive skills, which are often neglected by conventional procedures. Furthermore the visual search task is not susceptible to practice effects (at least in healthy individuals), predestinating this tool for the assessment of cognitive changes, i.e. due to disease progression or disease-modifying treatment.