The neurodegenerative disorder amyotrophic lateral sclerosis may render patients unable to speak or write, so that objective assessment of cognitive impairment, which is commonly of a dysexecutive nature, is challenging. There is therefore a need to develop other methods of assessment that utilize other relatively unaffected motor systems. In this proof-of-principle study a novel eye-tracking version of the trail-making test was compared with performance on the standard written version in a group of healthy volunteers. There was good correlation for speed between both versions of Part B (R2=0.73), suggesting that this is a viable method to objectively assess cognitive impairment in disorders where patients are unable to speak or write.
Citation: Hicks SL, Sharma R, Khan AN, Berna CM, Waldecker A, Talbot K, et al. (2013) An Eye-Tracking Version of the Trail-Making Test. PLoS ONE 8(12): e84061. doi:10.1371/journal.pone.0084061
Editor: Pedro Gonzalez-Alegre, University of Iowa Carver College of Medicine, United States of America
Received: May 15, 2013; Accepted: November 11, 2013; Published: December 18, 2013
Copyright: © 2013 Hicks et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This work was supported by the Rhodes Foundation (RS), Wellcome Trust (SH, CB, CK), Medical Research Council & Motor Neuron Disease Association Lady Edith Wolfson Clinician Fellowship (MRT), and the NIHR Biomedical Research Centre, Oxford (CK). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
The trail-making tests (TMTs) are neuropsychological assessments of visual attention, psychomotor processing speed, and task-switching. The test was developed from the Taylor Number Series Test, which required the subject to connect a randomly distributed set of numbers sequentially from 1 to 50 . The most widely used version of the TMT comprises parts A and B which are both timed in seconds to completion . In part A, the subject uses a pencil to connect a series of 25 encircled numbers in numerical order (1, 2, 3…). In part B, the subject connects 25 encircled numbers and letters in numerical and alphabetical order, alternating between the numbers and letters (1, A, 2, B, 3, C, 4, D…). The numbers and letters are placed in a semi-random fixed order, in such a manner as to avoid overlapping lines being drawn by the examinee. While part A is generally presumed to be a test of visual search and motor skills, part B is considered to be a test of higher level cognitive skills such as mental flexibility [3-5].
Electroencephalography has implicated frontal lobe activation during performance on the TMT , while a study using a verbal adaptation of the TMT  found that the set shifting component (1, A, 2, B, 3, C,...) activated the dorsolateral prefrontal cortex (DLFPC) and supplementary motor area, postulated to be sensitive to executive functioning, particularly cognitive flexibility. Functional magnetic resonance imaging demonstrated activation of the left DLPFC, precentral gyrus, cingulate gyrus and medial frontal gyrus during part B .
Amyotrophic lateral sclerosis (ALS) is a progressive neurodegenerative condition and has clinical, pathological and genetic overlap with frontotemporal dementia . Cognitive involvement in ALS is characterised by consistent deficits in executive function, which carry adverse prognosis . The recognition of ALS as a multi-system cerebral disorder has made neuropsychological assessment a routine part of clinical research, but its application is limited by the frequent loss of use of writing and speech functions as a result of disease progression. Motor neurons subserving oculomotor function, in contrast, are largely preserved throughout the course of ALS . The wider cerebral networks involved in saccade generation, particularly in the frontal lobes, in conjunction with the increased sophistication and practicality of eye-tracking equipment, make eye movements a potentially uniquely practical method of studying extramotor cerebral pathology in ALS. While language deficits , particularly verbal fluency , appear to be the most consistently impaired neuropsychological tests in ALS patients, TMT performance is also affected [14,15], and has been specifically linked to widespread frontal lobe cerebral white matter changes .
Among the wide range of neuropsychological tests sensitive to ALS pathology , the TMT was the most obviously transferrable to the hands-free, speech-free environment. We therefore developed an oculomotor-driven version of the TMT and compared it to the traditional written version in a group of healthy volunteers.
The study was approved by the London Riverside Research Ethics Committee (04/q/0406/60). Written informed consent was obtained from all participants, and all research was carried out in UK.
Forty healthy volunteers aged between 22 and 52 years (mean age 28±7) with no significant past medical or psychiatric histories were recruited (21 females, 19 males). Each participant performed both written and oculomotor versions of the TMT. To minimize practice effects, half of the subjects performed the written version of the test first, followed one week later by the oculomotor version, and vice versa for the other half.
The written TMT was administered according to standard protocol . Participants were instructed to draw a continuous line on the paper “quickly and accurately” connecting the stimuli. If a subject made a mistake during the test and did not correct it, the study investigator then immediately corrected them. The total time to connect all 25 items on both sheets (A and B) was measured, including the extra time required for the administrator to correct for errors.
A horizontal flipping of the written trail was used, keeping the arrangement of items intact and rotated by 90 degrees to accommodate the landscape screen format (resolution 1280 x 1024 pixels). The stimuli (letter or number) were included into contingent squares of 150 pixels. An EyeLink-1000 infrared pupil tracker with EyeLink Experiment Builder® software was used to record fixation accuracy for all participants throughout all experimental conditions. Eye movement data were analysed off-line using Experiment Builder (SR research) and custom programs written in Matlab. Head position was stabilized using a chin rest and the distance to the screen was 57 centimeters. A nine-points calibration covering the totality of the visual screen and a drift correction was carried out before each test (A/B) to ensure an accurate eye position recording.
In order to select an item on the computer screen, participants were required to fixate on the item for a minimum of 400 milliseconds. This delay was designed to avoid unintended activation during visual scanning. When the item was fixated, the ring surrounding the number or letter turned from white to red, as a signal for the participant to move to the next item in the sequence. Only the correct item in the sequence could be selected (and turn red), which made it possible to score an attempt to select an item as an error (see Video S1). Once an item was selected, the previous item in the sequence turned from red to dark grey to mark it as completed.
As per standard procedure in the written TMT, subjects were allowed to practice the oculomotor-based tests once immediately before the testing, using Parts A & B with fewer items and a different spatial arrangement. This also allowed subjects to experience the concept of gaze contingency (the online response of the program to eye position) and the fixation duration required to select the correct item (400ms) in a sequence.
To demonstrate a difference in cognitive load between Parts A & B, the number of fixations between targets, i.e. the number of different targets subjects gazed at before arriving upon the right target, was examined.
A linear regression analysis was performed on the data to determine whether scores on the written TMT could predict scores on the oculomotor TMT. There was good correlation between total time taken to complete the written versus oculomotor version of Part B (R2=0.73), though no correlation for Part A (R2=0.07) was revealed (Figure 1). There was no effect for age or gender in part A (p=0.74 and p=0.23 respectively), or part B (p=0.65 and p=0.44 respectively). The number of fixations decreased progressively for Part A (R2=0.41), but not in Part B (Figure 2).
There is a strong correlation for Part B (lower panel) but none for Part A (upper panel).
This study demonstrated that an oculomotor-driven version of the TMT can be applied to healthy volunteers, and in which Part B performs comparably to the standard written version. Notwithstanding the need for formal reliability and validation testing, it has potential to explore dysexecutive functioning in those unable to write or speak, and broadens the potential application of eye-tracking to encompass tests of cognitive function in ALS.
The striking disparity between the correlations of Trail A and Trail B between paradigms was unexpected, although this was also observed in a comparative study of the oral and written versions of the TMT . Like these authors, we attribute the lack of correlation between the written and oculomotor version of Part A to the low cognitive input required for this part of the test, so that variability due to individual preference for method (written versus eye-tracking) was more apparent. Part B has inherently high cognitive demands , that may mask any variance attributable to input methodology. This is supported by the progressive drop in fixations seen in Part A but not Part B of the oculomotor version. As participants progressed through the task, the number of available items decreased, reducing the number of fixations required to select the remaining targets. In Part A, this is seen as a steady reduction in oculomotor output, while in Part B the high number of fixations at the end of the trial may be the result of the increased cognitive load interfering with the participants’ search strategy. We were not able to quantify errors for the eye-tracking version of the test, though a systematic difference between the two methods seems unlikely.
Several variations of the TMT have been developed over the years. The Color Trails Test  was developed to accommodate testing in subjects with language difficulties, although there is some evidence that the Part B equivalent of this test is not compatible with the original TMT Part B, described as ‘culture-loaded’ because of its reliance on the Latin alphabet. It has been demonstrated that it is particularly unreliable in younger and less educated subjects .
An oral version of the TMT was developed and may distinguish between spatial and motor components from sequencing deficits when used in conjunction with the standard written test . It has been used in conditions like ALS where there is significant limb impairment, and it would be useful to directly compare the performance of the oral version alongside the eye-tracking one. Although novel analogues of the classic TMT have been found to be effective alternatives , in general computerized and paper-and-pencil versions of the test have not been found to be equivalent before now , possibly because of the variability of the motor demand of the computer interface . The gaze-contingent paradigm on which the experiment here is built minimizes the motor component. Furthermore, subjects are not expected to speak during the test either. Only eye movements are used to fixate and search targets on the screen in an intuitive way.
Portable eye-tracking devices have made a valuable contribution to the study of neurodegenerative diseases such as Huntington’s disease . Provided oculomotor functions are mechanically preserved, it is potentially possible to use this paradigm to test other cognitive domains in subjects who are unable to speak or write. The increased sophistication of eye-tracking equipment, means it is possible also to delineate the nature of errors that patients make while scanning targets, such as perseverative errors (the tendency to insist on staring at a selected item reflecting a lack of flexibility), errors due to disinhibition (the inability to maintain gaze for the required 400 ms) and sequencing errors (the inability to maintain two separate sets in working memory during Part B). This might allow more subtle exploration of cognitive dysfunction across a range of neurological disorders.
Video showing administration of the trail-making test Part B using the eye-tracking equipment. The subject’s head is apposed comfortably in the frame at a fixed distance from the screen. The target being fixated turns red and its border fades to grey to denote successful sequencing. As the camera pans to the left, the administrator’s screen is seen which shows the subject’s progress, followed by a screen showing the position of visual fixation at any moment as a white moving cursor.
Conceived and designed the experiments: SLH RS AW MRT. Performed the experiments: SLH RS ANK CMB AW. Analyzed the data: SLH RS CMB AW. Contributed reagents/materials/analysis tools: SLH CK. Wrote the manuscript: SLH RS ANK CMB AW CK KT MRT.
- 1. Army Individual Test Battery (1994) Manual of Directions and Scoring. Washington, DC: War Department, Adjutant General's Office.
- 2. Bowie CR, Harvey PD (2006) Administration and interpretation of the Trail Making. Test - Nat Protoc 1: 2277-2281. doi:10.1038/nprot.2006.390.
- 3. Crowe SF (1998) The differential contribution of mental tracking, cognitive flexibility, visual search, and motor speed to performance on parts A and B of the Trail Making. Test - J Clin Psychol 54: 585-591. doi:10.1002/(SICI)1097-4679(199808)54:5.
- 4. Arbuthnott K, Frank J (2000) Trail making test, part B as a measure of executive control: validation using a set-switching paradigm. J Clin Exp Neuropsychol 22: 518-528. doi:10.1076/1380-3395(200008)22:4;1-0;FT518. PubMed: 10923061.
- 5. Gaudino EA, Geisler MW, Squires NK (1995) Construct validity in the Trail Making Test: what makes Part B harder? J Clin Exp Neuropsychol 17: 529-535. doi:10.1080/01688639508405143. PubMed: 7593473.
- 6. Segalowitz SJ, Unsal A, Dywan J (1992) CNV evidence for the distinctiveness of frontal and posterior neural processes in a traumatic brain-injured population. J Clin Exp Neuropsychol 14: 545-565. doi:10.1080/01688639208402844. PubMed: 1400918.
- 7. Moll J, de Oliveira-Souza R, Moll FT, Bramati IE, Andreiuolo PA (2002) The cerebral correlates of set-shifting: an fMRI study of the trail making test. Arq Neuropsiquiatr 60: 900-905. doi:10.1590/S0004-282X2002000600002. PubMed: 12563376.
- 8. Zakzanis KK, Mraz R, Graham SJ (2005) An fMRI study of the Trail Making Test. Neuropsychologia 43: 1878-1886. doi:10.1016/j.neuropsychologia.2005.03.013. PubMed: 16168730.
- 9. Kiernan MC, Vucic S, Cheah BC, Turner MR, Eisen A et al. (2011) Amyotrophic lateral sclerosis. Lancet 377: 942-955. doi:10.1016/S0140-6736(10)61156-7. PubMed: 21296405.
- 10. Elamin M, Phukan J, Bede P, Jordan N, Byrne S et al. (2011) Executive dysfunction is a negative prognostic indicator in patients with ALS without dementia. Neurology 76: 1263-1269. doi:10.1212/WNL.0b013e318214359f. PubMed: 21464431.
- 11. Sharma R, Hicks S, Berna CM, Kennard C, Talbot K et al. (2011) Oculomotor dysfunction in amyotrophic lateral sclerosis: a comprehensive review. Arch Neurol 68: 857-861. doi:10.1001/archneurol.2011.130. PubMed: 21747027.
- 12. Abrahams S, Newton J, Niven E, Foley J, Bak TH (2013) Screening for cognition and behaviour changes in ALS. Amyotroph Lateral Scler Frontotemporal Degener.
- 13. Abrahams S, Leigh PN, Harvey A, Vythelingum GN, Grisé D et al. (2000) Verbal fluency and executive dysfunction in amyotrophic lateral sclerosis (ALS). Neuropsychologia 38: 734-747. doi:10.1016/S0028-3932(99)00146-3. PubMed: 10689049.
- 14. Hartikainen P, Helkala EL, Soininen H, Riekkinen P Sr. (1993) Cognitive and memory deficits in untreated Parkinson's disease and amyotrophic lateral sclerosis patients: a comparative study. J Neural Transm Park Dis Dement Sect 6: 127-137. doi:10.1007/BF02261006. PubMed: 8117409.
- 15. Kilani M, Micallef J, Soubrouillard C, Rey-Lardiller D, Demattei C et al. (2004) A longitudinal study of the evolution of cognitive function and affective state in patients with amyotrophic lateral sclerosis. Amyotroph_Lateral_Scler_Other Motor Neuron Disord 5: 46-54. doi:10.1080/14660820310017560.
- 16. Kolind S, Sharma R, Knight S, Johansen-Berg H, Talbot K et al. (2013) Myelin imaging in amyotrophic and primary lateral sclerosis. Amyotroph Lateral Scler Frontotemporal Degener.
- 17. Phukan J, Pender NP, Hardiman O (2007) Cognitive impairment in amyotrophic lateral sclerosis. Lancet Neurol 6: 994-1003. doi:10.1016/S1474-4422(07)70265-X. PubMed: 17945153.
- 18. Mrazik M, Millis S, Drane DL (2010) The oral trail making test: effects of age and concurrent validity. Arch Clin Neuropsychol 25: 236-243. doi:10.1093/arclin/acq006. PubMed: 20197294.
- 19. D'Elia L, Satz P (1989) Color Trails 1 and 2. Psychological Assessment Resources. Odessa, FLA.
- 20. Lee TM, Chan CC (2000) Are trail making and color trails tests of equivalent constructs? J Clin Exp Neuropsychol 22: 529-534. doi:10.1076/1380-3395(200008)22:4;1-0;FT529. PubMed: 10923062.
- 21. Ricker JH, Axelrod BN (1994) Analysis of an Oral Paradigm for the Trail Making Test. Assessment 1: 47-52. doi:10.1177/1073191194001001007. PubMed: 9463499.
- 22. Jacobson SC, Blanchard M, Connolly CC, Cannon M, Garavan H (2011) An fMRI investigation of a novel analogue to the Trail-Making. Test - Brain Cogn 77: 60-70. doi:10.1016/j.bandc.2011.06.001.
- 23. Drapeau CE, Bastien-Toniazzo M, Rous C, Carlier M (2007) Nonequivalence of computerized and paper-and-pencil versions of Trail Making. Test - Percept Mot Skills 104: 785-791. doi:10.2466/pms.104.3.785-791.
- 24. Kuhlman A, Little D, Sekuler R (2006) An interactive test of serial behavior: age and practice alter executive function. J Clin Exp Neuropsychol 28: 126-144. doi:10.1080/13803390590929289. PubMed: 16448981.
- 25. Antoniades CA, Xu Z, Mason SL, Carpenter RH, Barker RA (2010) Huntington's disease: changes in saccades and hand-tapping over 3 years. J Neurol 257: 1890-1898. doi:10.1007/s00415-010-5632-2. PubMed: 20585954.