Portable eye-tracking as a reliable assessment of oculomotor, cognitive and reaction time function: Normative data for 18–45 year old

Eye movements measured by high precision eye-tracking technology represent a sensitive, objective, and non-invasive method to probe functional neural pathways. Oculomotor tests (e.g., saccades and smooth pursuit), tests that involve cognitive processing (e.g., antisaccade and predictive saccade), and reaction time tests have increasingly been showing utility in the diagnosis and monitoring of mild traumatic brain injury (mTBI) in research settings. Currently, the adoption of these tests into clinical practice is hampered by a lack of a normative data set. The goal of this study was to construct a normative database to be used as a reference for comparing patients’ results. Oculomotor, cognitive, and reaction time tests were administered to male and female volunteers, aged 18–45, who were free of any neurological, vestibular disorders, or other head injuries. Tests were delivered using either a rotatory chair equipped with video-oculography goggles (VOG) or a portable virtual reality-like VOG goggle device with incorporated infrared eye-tracking technology. Statistical analysis revealed no effects of age on test metrics when participant data were divided into pediatric (i.e.,18–21 years, following FDA criteria) and adult (i.e., 21–45 years) groups. Gender (self-reported) had an effect on auditory reaction time, with males being faster than females. Pooled data were used to construct a normative database using 95% reference intervals (RI) with 90% confidence intervals on the upper and lower limits of the RI. The availability of these RIs readily allows clinicians to identify specific metrics that are deficient, therefore aiding in rapid triage, informing and monitoring treatment and/or rehabilitation protocols, and aiding in the return to duty/activity decision. This database is FDA cleared for use in clinical practice (K192186).


Introduction
Eye movements have been traditionally used as a non-invasive evaluation of neural functions, due to the wide representation of oculomotor nuclei throughout the brain [1,2]. Recent advances in eye-tracking technology using video-oculographic (VOG) recordings with high resolution cameras allow precise measurement and quantification of linear and torsional eye movements. The four types of eye movements-i.e., saccade, smooth pursuit, vergence, and vestibulo-ocular-can be assessed using a battery of tests that provide visual or motion stimuli to elicit these movements while recording horizontal, vertical, and torsional eye movements. Furthermore, the addition of cognitive (e.g., antisaccade, predictive saccade) and reaction time (e.g., auditory or visual) tasks permits probing of more complex neural processing. For example, antisaccade is a task in which the subject is required to look in the opposite direction of a stimulus appearing after fixation. This is a complex task comprised of several components: subjects must remember the task instructions, inhibit the natural tendency to look towards an appearing target, translate the spatial location of the target to its mirror image location, and execute a saccade to a blank position on the screen. The voluntary inhibition of the reflexive saccade is controlled by nuclei located in the frontal lobe; therefore, this test provides an estimation of injury or dysfunction of the frontal lobe [3].
Numerous studies have shown deficits in eye movements and reaction time in neurodegenerative diseases [4,5], including Alzheimer' disease [6], Parkinson's [7][8][9] and multiple sclerosis [10][11][12], psychiatric conditions [13], and more recently following mild traumatic brain injury (mTBI), also known as concussion [14][15][16]. In the past decade, awareness and understanding of mTBI/concussion has been substantially improved, along with a growing body of research uncovering acute and long-term effects of mild head injuries. It is now understood that it is important to accurately diagnose concussion in a timely manner, manage and provide adequate therapy, and decide when a person is able to return to duty/work/activity [14][15][16]. While clinicians are aware of this importance, simple tools for diagnosis and monitoring mTBI are still under development and evaluation.
Deficits in eye movements are present in up to 90% of patients with acute mTBI [17][18][19][20][21][22][23][24], thus positioning oculomotor tests as essential tools for evaluating mTBI. Although numerous studies have shown that oculomotor, cognitive, and reaction time tests combined with eye tracking can be beneficial in clinical practice, the adoption of these test in clinical practice is lacking for at least two reasons. First is the availability of a simple tool, preferably portable, with tests that are easy to administer and interpret. There are commercially available, FDAcleared, portable VOGs goggles equipped with high speed cameras and software that can be used both in clinical settings and on the sidelines of sports, but the technology is used mainly in research settings. Second is lack of standardized normative data, collected from healthy individuals, against which to compare patient results. Some studies do report oculomotor data from healthy individuals [19,[25][26][27]; however, often the number of subjects tends to be low or the age is not taken into account. In addition, data are reported usually in the format of mean and standard deviation, while in clinical practice, reference intervals are preferred. Establishing a normative database is critical to accurately assess oculomotor, cognitive, and reaction time deficits and functional performance outcomes. Such a database could serve diagnostic purposes and disease monitoring over time, thus aiding not only in diagnosis but also in evaluating treatment efficacy, rehabilitation, and return to duty/work/activity decisions.
The goal of this study was to construct an eye-tracking reference database, defined as data representing the range of performance on a particular test of a group of medically healthy individuals with homogenous demographic distribution [28] for oculomotor, reaction time, and cognitive tests in individuals 18-45 years old, to be used in clinical practice.

Study participants
All research activities were conducted according to the principles expressed in the Declaration of Helsinki, and were approved by the following Institutional Review Boards (IRB): a) University of Miami, IRB# 2015036; b) Naval Medical Center San Diego, IRB# NMCSD.2013.0060; and c) Madigan Army Medical Center, IRB# 393240-1. The trials were registered under NCT02486003 and NCT01832714. The study used written informed consent and only participants who signed the informed consent were enrolled. Participants consisted of 466 adult male and female volunteers, ages 18-45 years old (gender ascertained by self-report; see demographics in Table 1) recruited from three different sites: #1 University of Miami, Miami, FL (n = 166 subjects enrolled in 2015); #2 Naval Medical Center San Diego, San Diego, CA, (n = 50 subjects enrolled between 2013-2015); and #3 Madigan Army Medical Center, Fort Lewis, Washington (n = 250 subjects enrolled between 2013-2015). Participants were recruited from the general population and included non-professional athletes who participate in intercollegiate athletics, civilians, and military service members. This mixture ensures appropriate representation of different levels of activity and skills that are typically encountered in the general population. All three sites used the same inclusion/exclusion criteria listed in S1 Table in the supplemental data. Exclusion criteria included conditions/diseases that could impact the oculomotor, reaction time, and cognitive tests. Specifically, participants with history of brain injury, repeated blast exposure, presence of severe aphasia, history of diagnosed neuropsychiatric disorders (e.g., hypochondriasis, major depression, schizophrenia), neurodegenerative disorders, disorders of hearing and balance (e.g., Meniere's disease, multiple sclerosis, vestibular neuritis, vestibular schwannoma, sudden sensorineural hearing loss), cerebrovascular disorders, history of ear operation other than myringotomy tube in the past, and systemic disorders (e.g., chronic renal failure, cirrhosis of the liver) were excluded. Special populations including women who were pregnant, children under 18 year old, and those with impaired decision-making capacity were also excluded from study. Data presented here include 300 healthy controls subjects included in earlier papers from this laboratory that described the use of oculomotor, cognitive, and reaction time tests for evaluation of mTBI [17,21,29]. Subjects were categorized into two populations based on age ranges defined by the US FDA: pediatric subjects aged 18-21 years and adult subjects aged 22-45 years (Table 1).

Devices
All eye-tracking technology and software used in this study were developed by Neurolign USA LLC (formerly known as Neuro Kinetics, Inc., Pittsburgh, PA). Two FDA-cleared eye-tracking devices were used in this study: a) Neurolign Dx NOTC, formerly known as I-Portal 1 Neurotologic Test Center (NOTC) and b) Neurolign Dx 100, formerly known as I-Portal 1 Portable Assessment System™-Nystagmograph (I-PAS™). The study started using the NOTC initially and then later added the Dx 100 device to the protocol. The devices are substantially equivalent, i.e., both use the same software and have the same high-resolution eye-tracking capabilities. The only differences are in the type of stimuli that each device can deliver and the device physical size. The Neurolign Dx NOTC is a rotation chair that provides rotational, visual, and auditory stimuli to an individual while recording eye movements and reaction time responses. The Neurolign Dx 100 is a portable, compact, 3D, head-mounted display system with integrated eye-tracking that provides visual and auditory stimuli while recording eye movements. Motor reaction times are also recorded via a small, wired, hand-held trigger button box. Both the Neurolign Dx NOTC and the Neurolign Dx 100 use infrared video-oculography (VOG) and the same acquisition and analysis software. High resolution eye-tracking images were acquired via two high-speed digital infrared cameras (940nm; sampling rate 100 frames/sec). Spatial resolution for horizontal and vertical eye-tracking and torsion is <0.1 degrees; eye-tracking range is at least ±30 degrees horizontal, ±20 degrees vertical and ±10 degrees torsional. Data were collected using I-Portal1 software which captures, time stamps (critical for synchronization), and analyzes digital images of the eye, collecting horizontal and vertical eye movement data. VEST™ software was used to operate the hardware, manage and capture the stimulus profiles, integrate I-Portal1 eye-tracking data, and analyze data, generating a comprehensive set of metrics. Table 2 presents a detailed description of the test battery consisting of nine different tests to assess oculomotor, cognitive, and reaction time metrics. The described test battery has the ability to detect patterns of measurable deficits associated with mTBI and other neurological conditions [17-21, 30, 31], as well as oculomotor, vestibular, and neuro-otological conditions. The complete test battery was performed in 300 participants using the Neurolign Dx NOTC and in 166 participants using the Neurolign Dx 100. All participants were tested only once with the battery of tests listed in Table 2 and the following additional tests not included in this paper, the optokinetic response, subjective visual horizontal and vertical. In addition, participants from the Naval Medical Center San Diego and Madigan Army Medical Center were tested with sinusoidal harmonic acceleration, visual enhancement, visual suppression, and the computerized rotational head impulse test (crHIT) administered in the NOTC device. These data are included in a companion manuscript [32]. The order of the tests was the same for sites # 2 and # 3 (Naval Medical Center San Diego and Madigan Army Medical Center) and slightly different for #1 (University of Miami). In a previous pilot study [33] performed at a different site, we investigated the effects of the examiner and device on test results. Subjects (n = 30, healthy controls) were tested three times with varying time intervals between each session, ranging from 0.2h to 48h, using 3 different Dx 100 (I-PAS) devices and 5 different examiners. The order of the tests was different in each session. The results indicated that subjects' responses are not affected by either the examiner, the device or test order [33].

Data analysis
Acquired data for each test were inspected for completion and validity and analyzed in VEST™ software, with results exported for statistical analysis (see below). Treatment of artifacts and outlying samples. Data were filtered, or partially removed, on a test-by-test basis by manual adjustment of VEST™ software controls according to standard operating guidelines/procedures for the removal of artifacts (e.g., blinks, recording noise, temporary failures of eye-tracking, shifting of goggles, erroneous responses or those unrelated to the task) to separate eye movement signals from other recording noise, or to segregate saccadic activity from pursuit activity. Individual tests for some participants were removed from   Percentage predicted (%) = percentage of predicted saccades 6 Antisaccades: Subject is required to fixate on a central target for 1.5 to 2.5 seconds, after which a peripheral target is presented. Subject is required to generate an eye movement of the same distance as the target displacement, but in the exact opposite direction. There are 20 anti-saccades with time between saccades randomly selected from 1 to 2 seconds and random displacement between -24 to + 24 degrees.
Error Rate (%) = percentage of prosaccade errors, i.e. where the subject looks toward rather than away from the stimulus (Continued ) analysis when the data quality was judged to be inadequate for accurate measurement or produced analytic errors, or test was not run. Statistical analysis. Within each test, a number of metrics describing specific components of eye movement were calculated by VEST™ software and exported to IBM SPSS Statistics for Windows, version 21 (IBM Corp., Armonk, N.Y., USA) and Microsoft Excel for further analysis. The statistic results were reported based on VEST measurements.
Age and gender effect. To construct a normative database for the test variables (e.g., computing the reference intervals), we first examined the homogeneity of the participants' test results. In particular, we examined whether the test results are affected by the age and/or gender of the participants. For this purpose, the data was grouped by (a) age into two populations: pediatric subjects aged 18-21 years and adult subjects aged 22-45 years (age ranges defined by the US FDA), and (b) gender (ascertained by self-report): female and male. The effect of age and gender was tested using the t-test (mean difference) for independent samples and by applying one and two-way (with interaction) analysis of variance (ANOVA). The t-test and one-way ANOVA were used to examine each effect separately, whereas a two-way ANOVA was used to test jointly the effect of age and gender when the interaction between the two was considered as well (i.e., age group×gender). Levene's test for equal variance was used to decide whether to apply t-test that assumes equal variances or not.
Calculation of 95% reference interval (RI) with 90% confidence interval (CI). For each variable, a 95% RI with a 90% CI on the lower and upper limits of the RI was established assuming a non-parametric distribution, according to the FDA suggested guidelines and other published references [34,35]. In order to construct the RI, individual metrics were organized by rank from the lowest to the highest value: y 1 , y 2 , . . .y N , where N is sample size. The 100(1α)% RI is given by the upper and lower limits, R L and R U , calculated as follows: � with α = 0.05 (square brackets indicate the number is rounded to the nearest integer). For the 90% CI for the upper or lower limits of the RI, the 100(1-γ)% limits for CI, (γ = 0.1 for 90% CI), named R q and S q , for each R L and R U , were calculated [34,35]. The percentiles 2.5, 5, 10, 25, 75, 90, 95 and 97.5 for each test metric were calculated using a SPSS built in function.
One sided metrics. There are a number of metrics for which the lower or the upper limits are either meaningless or are not of clinical interest for the general population. For example, for saccade latency (i.e., the time it takes for the eye to reach the target), the lower limit, 2.5 th percentile, is not of clinical interest for the general population, because unusually fast saccades are likely not indicative of any clinical condition. The upper limit, however, is of interest because values greater than this number imply that the eye velocity to reach the target is very slow, which may have clinical implications. For these metrics, the 5 th or 95 th percentile was calculated for the limit of interest and the limit of no interest was marked with not applicable (n/ a) [34,35].
Methods for calculating 'Peak Velocity' in the saccade test. During a saccade, eye velocity corresponding to each eye displacement in response to a stimulus displacement was computed by VEST™ software and data were imported in Excel. Data from each subject were placed in bins of 5 deg based on eye displacement; i.e. saccades for which eye displacement was between 30 and 25 deg were grouped together, with the same for 25-20 deg, 20-15 deg, etc. Data were then treated as described above by ranking the values from smallest to the largest and calculating RI and CI as described above.

Results
Data presented here are from 466 participants, 307 males and 159 females with a mean age of 24.75 ± 6.27 (range 18-45 years) (see Table 1 for demographics). Previous research has shown that most oculomotor, cognitive, and reaction time metrics mature before age 18, are stable between 18-45, and decline thereafter [3,[36][37][38][39][40][41][42]. Therefore, this database was limited to this age range. The database includes 9 oculomotor, cognitive, and reaction time tests: horizontal and vertical saccades, horizontal and vertical smooth pursuit, predictive saccades, antisaccades, auditory reaction time, visual reaction time, and saccade and reaction time ( Table 2). From each test, a number of metrics describing different components of eye movement (e.g., amplitude, velocity, gain) were computed. The description of each metric is included in Table 2.

Age and gender effects
The US FDA considers the pediatric population to be 0 to 21-years-old, and the Journal of Academy of Pediatrics terms them as "Late Adolescents" [43]. Thus, pediatric participants (ages 18-21 years) and adult participants (ages 22-45 years) were analyzed to determine whether age has any effect on test results (Table 3). Gender (self-reported) was also examined ( Table 3). The findings indicate that with the exception of a few oculomotor, cognitive, and reaction time test metrics, the effect of age or gender was not statistically significant at the 0.05 level. However, with the exception of one test (namely the auditory reaction time test), the statistical effect does not suggest clinical significance, neither for age, nor for gender, because for each metric, the 95% RI with the 90% CI for each age group or gender group showed great overlap. For example, in the test 'Saccade-Random Vertical', for the metric 'latency', the 95% RIs with the 90% CIs were (sec): 0. 22  Analysis by self-reported gender indicated that gender had an influence on auditory reaction time, with males being significantly faster than females (Table 6).

Discussion
Tracking of eye movements is a well-established, non-invasive method for assessing many neurocognitive functions, and has been shown to be sensitive to detecting changes associated Table 3. Testing the effect of age group, gender, and interaction between the two on the test metrics using one and two-way analysis of variance (ANOVA). with traumatic brain injury, neurodegenerative diseases, and other dysfunctions [4,14]. Previous studies from our laboratories and others have demonstrated a role for oculomotor, cognitive, and reaction time tests in the detection, diagnosis, and monitoring of mTBI [17, 20, 21,  The upper and lower limits of the reference interval (RI) and 90% confidence interval (CI) for each limit are presented. For one-sided metrics, the limit of no interest is marked with not applicable (n/a). For description of each metric see Table 2.
These studies along with others [25,37,47], indicate the need for normative data for oculomotor, cognitive, and reaction time tests. The present study, by providing an FDA-cleared normative database, which reports the data in the format of reference intervals, may facilitate the adoption of these tests in clinical practice. The upper and lower limits of the reference interval (RI) and 90% confidence interval (CI) for each limit are presented. For one-sided metrics, the limit of no interest is marked with not applicable (n/a). For description of each metric see Table 2.
https://doi.org/10.1371/journal.pone.0260351.t006 The upper and lower limits of the reference interval (RI) and 90% confidence interval (CI) for each limit are presented. For one-sided metrics, the limit of no interest is marked with not applicable (n/a). For description of each metric see Table 2. https://doi.org/10.1371/journal.pone.0260351.t005

Neuro-anatomical pathways employed by oculomotor, cognitive, and reaction time tests
Oculomotor, cognitive, and reaction time tests probe the function of multiple neural pathways involved in the control of eye movement, vestibular function, reaction time, and motor and cognitive processing [2]. For example, saccades engage the frontal eye field area [1,2], reaction time tests engage networks in the superior temporal cortex, premotor cortex, and cerebellum [48], whereas antisaccades or predictive saccades engage additional executive function and attention networks [3,49]. Moreover, within each test, different components of planning and execution of eye or hand movements may engage different pathways. As such, saccade latency, which is the duration that it takes for the eye to initiate a movement towards the target, is controlled by central and cortical structures, while speed of eye movement and accuracy, i.e. how fast and how well the eyes land on the target, are controlled by both cortical and premotor structures at the brainstem level where the motor command is initiated [2]. The speed and amplitude of the saccades may involve the saccade burst generator located in the paramedian pontine reticular formation in the brainstem, which receives direct and/or indirect input from cortical structures (e.g., frontal eye field, parietal cortex), superior colliculus, and cerebellum [50,51]. Thus, our results, by providing detailed normative data for different components of eye (and/or hand) movements, may constitute the basis for understanding and evaluating disturbances of many neural pathways as a result of various neurological conditions.

Role of oculomotor, cognitive, and reaction time tests and normative database in mTBI
mTBI, also known as concussion, results in injuries to brain pathways that control or influence oculomotor behaviors, reaction time responses, and cognitive processes. In the last decade, a plethora of research has been dedicated to identifying oculomotor deficits in mTBI, and using oculomotor testing as an aid in diagnosis of acute mTBI [14]. These studies suggest that up to 90% of patients with acute mTBI present with deficits in one or more oculomotor, cognitive, and reaction time tests, such as deficits in saccades, smooth pursuit, reaction time, and tasks involving cognitive processing [17,19,21,24,52,53]. In addition, certain oculomotor deficits, e.g., saccades, reaction time, persist long after the initial injury [20,24,44]. Our previous work in mTBI indicated that a set of 6-8 metrics extracted from oculomotor tests, e.g., saccades, antisaccades, smooth pursuit, and vestibular tests, was strongly associated with the presence of a concussion in both acute and chronic patients [17,22]. These tests were also able to monitor progression over time after mTBI [21] and response to treatment [54]. For example, impairments in saccades (i.e., saccadic intrusions) following mTBI can cause visual disturbances (e.g., double vision, oscillopsia) accompanied by headaches. Improvements in saccadic eye movements using vision therapy (and/or medication) mitigated clinical symptoms [54]. This extensive research clearly highlights the usefulness of oculomotor testing in mTBI and other neurological condition. Thus, the availability of an oculomotor, cognitive, and reaction time normative database that is FDA-cleared, will have a significant impact in the adoption of oculomotor testing in clinical practice. This normative database eliminates the need for individual baseline pre-injury data, which is often unavailable, and instead provides the normative ranges for healthy individuals as a reference with which to compare patient data. In particular for mTBI, this database, as well as the normative database for vestibular tests [32], may aid clinicians in diagnosing, monitoring the recovery, making decisions regarding patient return to duty/work/activity, and assessing the effects of pharmacological treatments and/or rehabilitation protocols.

Conclusions
Recent advances in clinical eye-tracking technology, its non-invasive nature, the development of software and technology that provide precise and objective measures, and the batteries of tests that target not only the oculomotor systems, but importantly other motor, vestibular and cognitive functions (e.g., attention, cognition, decision making), propelled these methods as essential tools (biomarkers) for evaluating mTBI/concussion. The availability of an FDAcleared oculomotor, cognitive, and reaction time normative database aids clinicians in diagnosis, monitoring of treatment and rehabilitation protocols, and decisions concerning return to duty/work/activity. In addition, a normative database of this form has widespread applications not only for mTBI/concussion but also other neurological conditions, e.g., neurodegenerative diseases. The current limitation of this database is the age range, 18-45. Future studies are necessary to expand the database to other age ranges.
Supporting information S1