Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

What can entropy metrics tell us about the characteristics of ocular fixation trajectories?

  • Kateryna Melnyk ,

    Roles Conceptualization, Formal analysis, Investigation, Methodology, Software, Visualization, Writing – original draft

    k_m825@txstate.edu

    Affiliation Department of Computer Science, Texas State University, San Marcos, TX, United States of America

  • Lee Friedman,

    Roles Formal analysis, Investigation, Methodology, Software, Supervision, Visualization, Writing – review & editing

    Affiliation Department of Computer Science, Texas State University, San Marcos, TX, United States of America

  • Oleg V. Komogortsev

    Roles Funding acquisition, Project administration, Resources, Supervision, Writing – review & editing

    Affiliation Department of Computer Science, Texas State University, San Marcos, TX, United States of America

Abstract

In this study, we provide a detailed analysis of entropy measures calculated for fixation eye movement trajectories from the three different datasets. We employed six key metrics (Fuzzy, Increment, Sample, Gridded Distribution, Phase, and Spectral Entropies). We calculate these six metrics on three sets of fixations: (1) fixations from the GazeCom dataset, (2) fixations from what we refer to as the “Lund” dataset, and (3) fixations from our own research laboratory (“OK Lab” dataset). For each entropy measure, for each dataset, we closely examined the 36 fixations with the highest entropy and the 36 fixations with the lowest entropy. From this, it was clear that the nature of the information from our entropy metrics depended on which dataset was evaluated. These entropy metrics found various types of misclassified fixations in the GazeCom dataset. Two entropy metrics also detected fixation with substantial linear drift. For the Lund dataset, the only finding was that low spectral entropy was associated with what we call “bumpy” fixations. These are fixations with low-frequency oscillations. For the OK Lab dataset, three entropies found fixations with high-frequency noise which probably represent ocular microtremor. In this dataset, one entropy found fixations with linear drift. The between-dataset results are discussed in terms of the number of fixations in each dataset, the different eye movement stimuli employed, and the method of eye movement classification.

Introduction

In 2018, our research group introduced an extensive set of eye movement features [1]. This set includes features such as fixation duration, saccade peak velocity, and the percentage of saccades that were followed by post-saccadic oscillations (PSOs). Since then, this work has been used in oculomotor biometric applications by our group [25] as well as by other research groups [68]. Furthermore, these features have been used in various other applications, including the development of biomarkers [9], gender discrimination [10], and aging [11].

In the current study, we introduce an entirely new set of features based on various entropy measures. We applied these features to fixation trajectories, and in general, by using them, we address the question: “Which fixation trajectories are complex in some way, and which are not?”

The chaotic nature of eye movement trajectories was first evaluated 30 years ago [12]. As far as we are aware, no one has addressed the question: “What qualities of ocular fixation can be detected using chaos-based metrics?”

Chaos theory [1315] provides numerous metrics (see an overview Table 1) to assess different types of the complexity of an observed event. The chaos analysis developed rapidly and began to be applied in various biological [16] and social [17] science contexts. Chaos and entropy analysis have been applied to several forms of time series, including electrocardiograms (EKG) [18], electroencephalograms (EEG), and magnetoencephalography recordings (MEG) [19], as well as arterial blood pressure time series [20]. These methods were found to enhance the characterization [21] or diagnostic relevance of these signals [22, 23]. One interesting paper employed Independent Component Analysis (ICA) and Renyi Entropy to detect artifacts in EEG signals acquired during signal acquisition [24].

thumbnail
Table 1. An overview table presenting core chaos and entropy measures.

https://doi.org/10.1371/journal.pone.0291823.t001

Saccades

One interesting study [25] used chaos theory for the analysis of saccadic eye movements of healthy subjects. They assessed the complexity degree in the saccade dynamics by some computational tests (Power Spectrum, portrait in the state space, Fractal Dimension, Hurst Exponent, and Largest Lyapunov Exponent) and showed that there was a chaotic dynamic trend in saccade behavior. In their next study [26], they used metrics from chaos theory to evaluate saccades under two conditions: a saccade tracking task without distraction and one with distraction (cognitive load). The chaos of saccades was assessed with various metrics under both no-load and load conditions. It was found that the saccades during the no-load condition were more chaotic than saccades during the load condition.

Another study used a wavelet analysis of eye movements, collected during scene-viewing, to remove artifactual “spikes” that occur during signal acquisition [27]. Then they evaluated patients with cerebellar disorders as compared to healthy subjects. They found that Wavelet Entropy was a valuable measure of physiological motor noise.

Another group was interested in applying entropy measures to a visual scanning task to assess “visual scanning efficiency” [28]. The equation for Shannon Entropy was used to create the metric “Stationary Gaze Entropy” (SGE). The equation for Conditional Entropy was used to create “Gaze Transition Entropy” (GTE). They present a model of gaze orientation, with GTE as a measure of visual scanning efficiency and SGE as a measure of gaze dispersion.

This same group of authors [29, 30] looked at the usefulness of SGE and GTE to assess visual scanning while intoxicated. In the first study [29] they studied the effect of alcohol on gaze behavior and on measures of entropy. In the later paper [30] the authors evaluated whether alcohol’s effect on gaze behavior can be detected during a simulated driving task. They found that as blood alcohol level was either increasing or decreasing there were statistically significant changes in SGE and GTE. The fixation rate was not affected by these conditions, however.

In [31], the authors measured the optimum phase space parameters (τ, and m) for a series of eye movement velocity recordings collected while subjects tracked a jumping point target. The velocity records included the saccade to a new target and the fixation on that target. Recordings were obtained for unfiltered data and data filtered by one of 7 low-pass filters. Larger τ values and m values were associated with more restrictive filters. The authors also found evidence of a long-range correlation in eye-tracking data. This can be interpreted as indicating that the signal is “self-similar” and can be considered a fractal signal.

Two papers focused on the usefulness of chaos and entropy-related features in eye movement classification [32, 33]. Harezlak et al. [32] reported that Approximate Entropy on various time scales might be useful for saccade detection. In a follow-up report [33] they found that Approximate Entropy, Fuzzy Entropy, and the Largest Lyapunov Exponent were more useful in eye movement classification. However, the percent correct classification in the best case was in the low 80%. This is probably not competitive with current state-of-the-art eye movement classification systems [34].

Fixations

In the eye-tracking field, a number of reports have already evaluated chaos- and entropy-related metrics from fixational eye movements [35]. Their database consisted of fixations of three lengths, 1000, 1500, and 2999 ms duration. They investigated the correlation dimension (CD) of these 3 fixation durations before and after noise removal by a digital low-pass filter (50 Hz cutoff) of the eye movement signals. They found that the CD decreases with increasing fixation duration. They also found that the CD for the 1000 and the 1500 ms durations increased for low-pass filtered signals.

One paper [36] focused on fixations during the viewing of a complex scene. They did not analyze fixation trajectories. Rather, they examined the relationship between the dispersion parameter used in some fixation detection algorithms and the number of fixations detected. Obviously, as the dispersion threshold increases, fewer fixations are found. This relationship follows a power law. From this relationship, they present a method to measure the fractal dimensionality of a fixation. They note that the scanning patterns of typically developing toddlers may have fractal qualities.

Fixation eye movements from patients with schizophrenia were compared to those from a normal control group during reading [37]. The eye movement signals themselves were not analyzed. Rather, the focus was on the distribution of fixation durations. The authors found that the cumulative distribution of fixation durations from the patient group was a linear curve, suggesting that these fixations are a “temporal fractal”. This was not found for the control group.

In [38], the researchers studied the relationship between images with various levels of complexity (higher “fractality”) and the fixations recorded while viewing them. They found that there was a significant effect of the stimuli and that stimuli with higher complexity (higher fractality) cause fixational eye movements with lower fractality.

In [39], the authors evaluated both power spectral density analysis and fractal dimension determination from reading eye movements. They make two conclusions: (1) the underlying mechanism for eye movements might be effectively modeled by a chaotic, nonlinear, dynamical system possessing a strange attractor with a non-integer dimension. (2) The correlation dimension of the attractor, D2, an index of the complexity of the motion, seems to be related inversely to the qualitative assessment of reading ability.

Scanpaths

Scanpaths are eye movements collected by a gaze-tracking device, where information is recorded about the trajectories (paths) of the eyes when scanning the visual field (areas of interest or AOIs) and viewing and analyzing any kind of visual information. Many studies have related scanpaths to cognitive abilities such as learning, discrimination, and recognition of AOIs [4043]. Some of this research has indicated that scanpaths are essential to visual learning and recognition [44, 45]. Entropy-related metrics have been applied to evaluating scanpaths [4651].

Entropy measures of scanpaths have been used in studies of autism, attention-deficit/hyperactivity disorder (ADHD), and anxiety. In autistic patients, entropy metrics have been used to evaluate scanpaths during face recognition [47]. The researchers evaluated patients with autism and healthy controls and found the entropy metrics suggested different developmental trajectories for the two groups. In another study, patients with ADHD were compared to a group of age-matched typical children [48]. Scanpaths were evaluated as subjects viewed words they were instructed to remember. The ADHD group did not follow typical scanpaths while viewing the words. Rather, their visual scanning was discontinuous, uncoordinated, and chaotic. In another study [49], subjects had to land an airplane in a simulator during normal conditions and under various types of anxiety. Visual scanning entropy, which is the predictability of visual scanning behavior, showed an increase in the randomness of scanning behavior when anxious.

The creators of marketing material often use particular visual cues or stimuli to bias the gaze toward a specific direction. This is referred to as gaze guidance. In the work of Hooge and Camps [50], a measure of scanpath entropy was able to assess the effectiveness of gaze guidance [50].

Krejtz et al. [51] evaluated the usefulness of normalized Shannon Entropy during gaze transitions between AOIs. This metric expresses the complexity of both transition and stationary distributions between AOIs. They showed that higher transition entropy denotes more randomness and more frequent switching between AOIs. Also, they showed that higher stationary entropy indexes the extent to which individual visual attention is distributed among AOIs. Thus, they conclude that entropy measures are promising eye movement indicators of individual differences in curiosity, interest, and AOI familiarity.

In the present study

Most prior research has focused on only a small set of chaos and entropy metrics. No prior report has evaluated several entropy metrics simultaneously for fixation time series specifically and exclusively. In the present study, we evaluated six entropy metrics as applied to fixation trajectories. For fixation eye movement signals, we employed three datasets: (1) GazeCom [52, 53]; (2) the “Lund” dataset [54], and a dataset from our laboratory (“OK lab”) [55]. In our Methods section we endeavor to impart a more intuitive understanding of what aspect of a fixation signal that each entropy metric is sensitive to. To evaluate the types of fixations that high and low values of our metrics found, for each entropy measure we evaluated a page of 36 fixations with the highest entropy values and a page with 36 fixations with the lowest entropy values. Our overall aim is to determine what entropy metrics can inform us about fixation trajectories. With three datasets, we can evaluate if this information is dataset-specific.

Methods

Description of the GazeCom dataset

The GazeCom data set consists of eye movement recordings at 250 Hz from an SR Research EyeLink II device.

From [53], who employed the GazeCom dataset:

The GazeCom dataset consists of 18 clips (videos), around 20s each, with an average of 47 observers per clip (total viewing time over 4.5h). The total number of individual labels is about 4.3 million (including the samples still recorded after a video has finished; 72.5, 10.5, 11, and 5.9% of all samples are labeled as parts of fixations, saccades, pursuits, or noise, respectively). Event-wise, the data set contains 38,629 fixations, 39,217 saccades, and 4,631 smooth pursuits.

In this dataset, the horizontal position is measured in degrees of visual angle from 0 to 50 degrees. Zero represents the extreme left of the screen, 50 represents the extreme right of the screen and 25 degrees represents the left-right center of the screen (Mikhail Startsev, personal communication).

A full description of the dataset, the purpose of the dataset, the types of stimuli, and the recording details are provided in [52] and also [53]. We chose GazeCom, in part, because of the large number of identified fixations.

We derive our information as to how saccades were classified in Gazecom by reference to several papers [52, 53, 56]. Here is how [52] describes the process:

To initialize the search for a saccade onset, velocity had to exceed a relatively high threshold (138 deg/s) first. Then, going back in time, the first sample was searched where velocity exceeded a lower threshold (17 deg/s) that is biologically more plausible but less robust to noise (both parameters were determined by comparing detection results with a hand-labeled subset of our data). In a similar fashion, saccade offset was the first sample at which velocity fell below the lower threshold again. Finally, several tests of biological plausibility were carried out to ensure that impulse noise was not identified as a saccade: minimal and maximal saccade durations (15 and 160 ms, respectively) and average and maximum velocities (17 and 1030 deg/s, respectively).

Here is some additional information from [56]:

The two novice annotators were paid undergraduate students who received basic instructions about eye movements and interpreting eye tracker data. Experts in the eye movement field were available to answer their questions at any point in the labeling process. Due to their little prior experience with hand-labeling and because we wanted their internal biases to stabilize, these two annotators went through the data set for a second time several months later. In the first pass, they were provided with the prelabelled suggestions from the automatic method just described and were instructed to change, add, or remove intervals accordingly. In the second pass, they were presented with their own labeling and instructed to change it wherever they thought it was not accurate (with respect to the eye movement definitions). As a quality assurance measure, a third (expert) annotator (one of the authors) re-examined all the recordings in the data set with the objective of resolving conflicts between the labels of the first two annotators, also making changes where the provided eye movement definitions were violated.

It is worth noting that the post-saccadic oscillations (PSOs) were not classified separately in the GazeCom dataset, and yet, PSOs do occur in this dataset. In such a case: (1) the PSO is considered part of the saccade, or (2) the PSO is considered part of the fixation, or (3) some part of the PSO is considered part of the prior saccade and some part of the PSO is considered part of the following fixation. We note when a fixation begins with PSO or a part of a PSO.

One fixation was lost in the interpolation step (described below). This left 38,628 fixations. The median duration for a fixation was 60 samples. We also rejected fixations that were longer than an outlier threshold determined by the generalized extreme Studentized deviate test [57]. On this basis, fixations longer than 286 samples were outliers (N = 711). Of the remaining fixations, 831 fixations were rejected because there were not 30 samples of data before the fixations and 876 were rejected because we did not have 30 samples of data after the fixations. For our entropy calculations, we needed more than 10 samples. A number of fixations (N = 95) were rejected because they were less than 11 samples in length. During calculation of the entropies, if any fixation produced any entropy value less than or equal to 0.0, NaN (“Not a number”) or was infinite, the entire fixation was dropped from the analysis (N = 465). After these screening steps, there were 35,650 fixations available. We present a violin plot of all fixations lengths, in samples, in Fig 1(A).

thumbnail
Fig 1. Fixation durations across the three datasets.

In Fig 1 violin plots are presented representing the duration of fixations (in samples) across the three datasets. The durations between datasets were statistically significantly different. In Fig 1B we show the same plots after we had taken steps to equalize the fixation duration distributions across datasets.

https://doi.org/10.1371/journal.pone.0291823.g001

Although the nominal sample rate of the GazeCom dataset is 250Hz (an inter-sample interval (ISI) of 4 ms), there are very many unique ISIs other than 4 ms. Of all ISIs (4,227,557) 93.9% had the correct ISI (4 ms). Prior to the analysis of a dataset, the time stamps, horizontal position values, and classification codes were interpolated to a 4 ms ISI.

Description of the Lund dataset

What we refer to as the “Lund” dataset was first described in Larsen et al. [54]. It used to be publicly available but we don’t know if it still is. Interested users may want to communicate with Marcus Nyström (email: marcus.nystrom@humlab.lu.se).

This dataset was employed by Zemblys et al [58]. Here is how these authors described the dataset:

1cm “It consists of monocular eye movement data of participants viewing images, videos, and moving dots. The eye movements of all participants were recorded with the SMI Hi-Speed 1250 eye-tracker, running at a sampling frequency of 500 Hz. Two domain experts then manually segmented data into fixations, saccades, post-saccadic oscillations (PSO), smooth pursuit, blinks, and undefined events. A comprehensive description of the Lund dataset and the coding process can be found in [54].” (page 3, left column, last paragraph).

The two domain experts that classified this data were Marcus Nystrom and Richard Andersson, both considered experts in the classification of eye movements. There were 34 recordings where subjects were observing static images for 10 seconds. Although the description of the dataset states that the dataset was collected with a sampling rate of 500 Hz, Friedman [59] reported that 6 of the recordings were actually collected at 200 Hz. After the removal of these recordings, 28 recordings were left. The largest number (16) of good datasets (i.e., collected at 500 Hz) were classified by Andersson. Since we were not interested in comparing raters, we only evaluated these 16 recordings.

These recordings were classified into 6 categories: (1) Fixation, (2) Saccade, (3) Post-saccadic oscillation, (4) Smooth pursuit, (5) Blink, and (6) Undefined, i.e., any sample that does not fit any of the other categories. Since GazeCom was collected at 250 Hz, we downsampled all fixations in the Lund dataset to 250 Hz.

Description of the OK Lab dataset

The eye tracking database from our group is fully described in [60] and is labeled “GazeBase”. All details regarding the overall design of the database, subject recruitment, tasks and stimuli descriptions, calibration efforts, and eye-tracking equipment are presented there. There were 9 temporally distinct “rounds” over a period of 37 months, and round 1 had the largest sample. This report only includes 19 subjects. We started with 20 but dropped one very low-quality recording. The subjects were randomly chosen from 322 subjects available from round 1. Briefly, subjects were initially recruited from the undergraduate student population at Texas State University through email and targeted in-class announcements. Subjects completed two sessions of recording (median 19 min. apart) for each round of collection. Each session consisted of multiple tasks. The only task employed in the present evaluation was the text reading task. This smaller dataset of 19 subjects recorded while reading the text we refer to as the “OK Lab” dataset. Each subject was asked to read, silently, an identical pair of quatrains from the famous nonsense poem, “Hunting for a Snark”, written by Lewis Carroll (written from 1874-1876). The text was displayed in Times New Roman 20 pt. size bold font and was single-spaced. The mean letter interval for each piece of text was approximately 0.50 degrees of the visual angle. The height of the line of the text was 0.92 degrees of visual angle. Monocular (left) eye movements were captured at a 1,000 Hz sampling rate using an EyeLink 1000 eye tracker (SR Research, Ottawa, Ontario, Canada).

The initial purpose of the OK Lab dataset was a study of inter-rater agreement [55]. In that study, fixations were coded as “1”, saccades as “2”, and PSOs as “3”. All other classified events (blinks, unclassified data, forms of noise, and artifacts) were also coded. There were 4 raters. A very detailed set of manual classification rules was created to guide the raters. This manual is available at https://digital.library.txstate.edu/handle/10877/13373. Rater 1 was the best-performing rater, and so, in this paper, we only use the classifications made by rater 1.

Since the dataset was collected at 1000 Hz, we downsampled the fixation signals to 250 Hz to be more comparable to GazeCom.

For all entropy illustration figures, the data points represent samples from a time series, spaced 2.5 ms apart. Additionally, the horizontal position is measured in degrees of visual angle.

Fixations durations for the three datasets

Initially, the fixation durations for the three datasets were statistically significantly different (see Fig 1(A)). The GazeCom dataset had 35,650 fixations, the Lund dataset had 409 fixations and the OK Lab dataset had 1,852 fixations. Before comparing the datasets on entropy measures we wanted to equalize the distributions of fixation durations across the datasets. The first step was to remove all fixations below 20 samples and above 80 samples. To further equalize the distributions, we randomly removed some higher duration fixations for both the GazeCom and the Lund datasets. The final duration violin plots are presented in Fig 1(B). Note that the medians are now very close and that the distributions are no longer statistically significantly different. After equalization, the GazeCom dataset had 20,613 fixations, the Lund dataset had 145 fixations and the OK Lab dataset had 874 fixations.

Description of software used for calculations

For this report we used the toolkit “EntropyHub” (Please visit the website www.EntropyHub.xyz to see the guidelines for using EntropyHub.) that provides a wide range of functions to calculate different entropy statistics. EntropyHub is licensed under the Apache License (Version 2.0) and is free to use by all [61].

We originally evaluated 15 different entropy metrics in a pilot study. Based on the inter-correlations among the 15 metrics and the apparent relevance of each metric for discriminating between fixations, we chose six relatively uncorrelated and informative metrics for the formal study. In Table 2 we present the 6 entropy measures we chose for our analysis.

In Table 3 we present a list of math symbols that will be used throughout the paper. We also include some information about phase space parameters that will be explained below.

Phase space reconstruction

Introduction to phase space.

Since the three features described below employ the concept of phase space, this concept will be described here. Generally, the one-dimensional time series X with a sampling rate T given by the biological system can be mathematically represented as a system of differential equations [62]. Some components of the eye movement signal have been modeled at this time [63, 64]. However, no set of such equations exists to model the full complexity of entire eye movement signals.

By applying specific phase space reconstruction techniques from embedding theory [31], it is possible to reconstruct the time series into a high dimensional vector space and evaluate the complexity of the signal. For the reconstruction of the original system dynamics, we need the two embedding parameters [65]: τ time lag and m, the embedding dimension.

Embedding parameters for the phase space reconstruction.

The initial time series X, with measurements equally spaced in time x(1), x(2), …, x(N), can be reconstructed into a 2-dimensional matrix (Y) with rows representing states of the system and columns representing the coordinates of these states. In other words, the initial signal X is said to be embedded in a new matrix Y such that f(X) = Y.

Time lag, τ.

The embedding parameter τ [66] is a period of time between two observations (samples) x(i) and x(j) such that j > i. The general formula for the time series X given τ is: (1)

For example, if τ = 4, then: (2)

With this parameter, we can process a set of time series with different durations between samples.

Embedding dimension, m,.

The embedding parameter m [67] is the dimension of the space for the reconstruction of a phase portrait of X, such that X in YRm, where Rm is a m-dimensional euclidean space. Accordingly, we formally define our procedure as f: XYRm.

Phase space construction.

We construct a set Y of probable state variables from the initial signal X. Y is a matrix of possible states. Every possible state y(i)∈Y is represented by a delayed measure of X. Function f: XY is called a delay embedding procedure. As noted above, the two input parameters of f are embedding parameters (τ, m).

There are many potentially different phase space reconstructions (Y) for a given X time series, each characterized by different embedding parameters [68]. The X-sequence of length N will be divided into the sequence of vectors Y of length L where the value of L can be defined as L = N − (m − 1)τ. In other words, if the signal time series has a length of 23, m = 3, and τ = 4, the length L of Y will be 15.

Finally, Y will be in the next form: (3) (4) where each vector y(i) has length m and time difference between elements in vector y(i) is τ.

In our analysis, described in detail below, we create one phase space per fixation with parameters τ = 1, m = 2.

Illustrate phase space reconstruction.

To further illustrate phase space reconstruction, we present Tables 4 and 5. Table 4 contains an actual 23 sample fixation (horizontal position). We employ the MATLAB routine “phaseSpaceReconstruction” which takes as input the signal and produces the optimized estimated τ (τ = 4 in this case), using average mutual information [69]), the optimized embedding dimension m (m = 3 in this case, using the False Nearest Neighbor algorithm [70], and the phase space reconstruction in Table 5. Note that a number that is the 3rd coordinate of the 1st state is identical to the 2nd coordinate of the 5th state which is itself identical to the 1th coordinate of the 9th. These differences in position represent the effect of τ = 4, the time lag.

thumbnail
Table 5. Phase space reconstruction for the signal in Table 4 to the left.

https://doi.org/10.1371/journal.pone.0291823.t005

In Fig 2, we plot the phase space (in Table 5) for this particular fixation period (in Table 4).

thumbnail
Fig 2. Plot of the trajectory of the states thru the phase space from Table 5.

https://doi.org/10.1371/journal.pone.0291823.g002

Entropy-related features

Shannon Entropy: A fundamental measure employed in various entropy algorithms.

By definition, “Shannon Entropy” (SE) measures the amount of information H(X) contained in a continuous time series of events X [71]. If our time series is simple, it will be characterized by low entropy; whereas, if it is complex, it will exhibit high entropy. For the discrete random variable X with possible outcomes x1, x2, …, xN we create, a series Y where every state ym(i) is presented as a pattern with m samples. We first need to determine if each state is unique or if some states are repeated, and associate every unique state with the probability that describes its occurrence in the phase space. Once we have a set of probability values, SE can be formally defined as: (5) where Pm stands for the joint probability of the pattern ym(i) and the sum is extended to all the different ym(i) patterns [72].

Entropy features that require phase space reconstruction.

Metric 1: Fuzzy entropy. In 1965, Zahed [73] introduced the mathematical concept of a “fuzzy set”. A fuzzy set characterizes the input-output relationship in an environment of imprecision. Fuzzy Entropy(FuzzEn) is based on this concept. As explained below, the fuzzy function produces a measure of the similarity between the two states, based on their shape (trajectory) in phase space.

For the N-sample series X we apply the phase space reconstruction technique with (τ, m). We generalize every state as follows: (6) where X-values are generalized by subtracting the mean of the signal [74]: (7)

For a given vector y(i), we define the distance between vector y(i) and another vector y(j) as the maximum absolute difference of the corresponding scalar components (coordinates) [74]: (8)

To evaluate the relationship between two vectors y(i) and y(j), we use the concept of “membership degree” from [73] with a fuzzy function μ [74] that associates each point y(i) with a real number in the interval [0, 1]. Following Zadeh’s work [73], we estimate the degree of similarity for a particular state y(i). The nearer the value of the fuzzy function is to unity, the higher the degree of similarity. Using n and r values, where n is a gradient of the boundary of the exponential function (estimated by the algorithm), we calculate the “membership degree” Dij of y(i) to y(j) through a fuzzy function as: (9)

Then we create the variable M = Nm and define the function ϕm for the two embedding dimensions equal to m and m + 1: (10) (11)

Finally, using Eqs (10) and (11), we determine the FuzzEn [74] of the finite dataset X as the negative natural logarithm of the deviation ϕm from ϕm+1: (12)

Illustration example: Fuzzy entropy. In this subsection, we illustrate the FuzzEn for 3 fixations, one with low FuzzEn, one with intermediate FuzzEn, and one with high FuzzEn. These plots all employ m = 1. As a reminder, we need to construct two phase spaces with m = 1 and m = 2 to calculate FuzzEn(1, n, r).

FuzzEn employs “fuzzy functions” which assess the similarity, or “membership degree” of the states, based on their shape (trajectory) in phase space. The input to each fuzzy function is the distance between two states. If the distance between the states is high, the membership degree produced by the fuzzy function is low. We noticed that for the fixations without any relatively large state-to-state transitions, the membership degree will be high. However, when the fixation trajectory contains very large state-to-state transitions, the membership degree will be very small. This will be reflected in the final entropy calculation.

In Fig 3, we see that the FuzzEn is higher for the fixations in Fig 3A and 3C than for the fixation in Fig 3E. This is because the fixations in Fig 3A and 3C both contain brief but relatively large eye position changes. Whenever we have large position differences in a fixation, we will also have large state distances in the m = 2 phase space (Fig 3B and 3D). The fixation in Fig 3E does not exhibit any noticeably large eye-position transitions, resulting in a low value for FuzzEn.

thumbnail
Fig 3. Explanatory plot for the fuzzy entropy.

In Fig 3A we have a fixation with high position change and relatively high FuzzEn. In Fig 3B we see the phase space reconstruction with the parameters τ = 1 and m = 2 for the fixation in Fig 3A and provide the membership degree for the trajectory in Fig 3B. In Fig 3C, we present a fixation with smaller position change and its phase space reconstruction in Fig 3D. In Fig 3E, we present a fixation with the lowest FuzzEn value and its 2-D trajectory in Fig 3F.

https://doi.org/10.1371/journal.pone.0291823.g003

Metric 2: Increment entropy. For the first step in the calculation of Increment entropy (IncrEn) [75], we use the time series X of length N to calculate the vector of sample-to-sample differences Υ of length N − 1. We refer to this vector of sample-to-sample differences as the “incremented X-sequence”, Υ: (13) where υ(i) = x(i + 1) − x(i). Next, we create a phase space from the incremented X-sequence, Υm, where m ≥ 1. We characterize the sign of the change between the one coordinate and its neighbor coordinate as either -1, 0, or 1. We do this for each pair of adjacent coordinates. The magnitude of the change between adjacent coordinates is calculated as follows: (14) where qr is referred to as the quantifying resolution [75]. At this point, each change between adjacent coordinates for each state can be summarized by a 2-symbol code [sign, magnitude]. Each state can be represented as a series of symbol codes (m − 1) ⋅ 2.

Next, we need to count the frequency of occurrence of each state’s pattern. The probability of each state pattern is the frequency of occurrence of the state pattern divided by the length of the time series. Plugging these probabilities into SE Eq (5) we get IncrEn.

Illustration example: Increment entropy. In Fig 4 we have the chart for Increment Entropy (IncrEn). As we can see, the first 3 examples from the top row (Fig 4A to 4C) are very rough fixations with complex trajectories. In the middle row, the trajectories are smoother, but they are still somewhat complex with fewer marked peaks and valleys. For the bottom row, the fixation trajectories are smoothest without any noticeable valleys or peaks.

thumbnail
Fig 4. The Increment Entropy.

The top row illustrates 3 fixations with high IncrEn, the middle row illustrates fixations with intermediate IncrEn and the bottom row illustrates fixations with low IncrEn.

https://doi.org/10.1371/journal.pone.0291823.g004

For the IncrEn algorithm, the sample-to-sample differences (referred to as the incremented time series) are used, and the phase space reconstruction is applied to this incremented time series. For every state, a “symbol pattern” is created by using the sign and the magnitude of change between the adjacent coordinates in each state. So, for fixations with many changes in sign and magnitude (see Fig 4A–4C), the IncrEn value is large. For fixations with fewer sign and magnitude changes the IncrEn is somewhat lower (see Fig 4D–4F). If there are few sign and magnitude changes, the IncrEn value is low (see Fig 4G–4I).

Metric 3: Sample entropy. Sample Entropy (SampEn) is considered an improvement over Approximate Entropy (ApEn) [76, 77]. The value of ApEn is dependent on the data length, i.e., the longer the length, the higher the ApEn. SampEn is more accurate and less dependent (although still dependent, see below) than ApEn on data length. For the SampEn, we use the same phase space reconstruction technique to rebuild a sequence X into a sequence Ym in real m-space Rm. We also use the Chebyshev formula to calculate the distance , ij between every pair of Y-states. For the fixed parameters m and r, we determine the two values A and B such that: (15) (16)

As a result, the SampEn will be determined as: (17) where the number can be considered as the conditional probability that two sequences within a tolerance r for m points remain within r of each other at the next point [78].

Illustration example: Sample entropy. Sample Entropy uses phase space reconstruction. This produces a matrix whose rows are states and whose columns are coordinates. The number of columns is determined by the embedding dimension (m). We used the default value m = 1. In this case, each state consists of sequential positions in the raw signal and can be thought of as existing in a 1-dimensional space. For the computation of this entropy, we start with a list of distances, specifically Chebyshev distances, between each state in 1-dimensional space. Sample Entropy considers the Chebyshev distances between all possible states and calculates the number of pairs for which the distance is smaller than, or equal to the fixed radius distance r. We refer to this count of distances less than r as B. In the default case, r = 0.2 * std(X). The next step in the calculation of Sample Entropy is to create a new phase space using m = 2 (a 2-dimensional space). Once again, the distances between all possible pairs of states are computed and the number of distances less than or equal to r is counted, we refer to this number as A. Sample Entropy is (18)

In Fig 5, we present a chart to help explain Sample Entropy. We have chosen 3 fixations with very similar threshold r values. This also means that we have chosen 3 fixations with very similar STDs. In Fig 5A, we present a very rough fixation in which a small percentage of the distances are less than r. In Fig 5B we present a 2-D fixation trajectory in which a very small number of distances are less than r. In the case of Fig 5A and 5B, the ratio of A/B in percent terms is 0.30. This produces a sample entropy of 1.26. In Fig 5C we present a smoother fixation (at least smoother than Fig 5A) with a larger percentage of distances less than r. In Fig 5D, for the 2-dimensional trajectory, a somewhat larger percent of distances is less than r. For this pair (Fig 5C and 5D), the ratio of A/B is 0.50, a somewhat larger ratio than for Fig 5A and 5B. This leads to Sample Entropy of 0.73, which is 57% less than the entropy for Fig 5A and 5B pair. In Fig 5E, we present a fixation that, with the exception of the saccade-like portion at the start, is very smooth. Not surprisingly, a very large percent of distances were less than a threshold. In Fig 5F, the percent of distances less than the threshold (r) is also relatively large. In this case, the ratio of A/B is almost 1.0, substantially larger than the other pairs. This is associated with a Sample Entropy of 0.26.

thumbnail
Fig 5. Explanatory plot for the Sample Entropy.

In Fig 5A we have a relatively chaotic fixation. In Fig 5B we see the phase space reconstruction with the parameters τ = 1 and m = 2 for the fixation in Fig 5A. In Fig 5C, we present a much less chaotic fixation and its 2-D fixation trajectory Fig 5D. In Fig 5E, we present a fixation with the lowest SampEn value and its reconstructed trajectory Fig 5F.

https://doi.org/10.1371/journal.pone.0291823.g005

0.0.1 Entropy features that do not require phase space reconstruction.

Metric 4: Gridded Distribution Entropy. Initially, with a constant sampling rate of 250 Hz, X is normalized within the range [0, 1] by a min-max normalization [79]: (19)

Next, the gridded Poincaré plot [80] is constructed for Xnorm. For each potential n horizontal grid lines and n vertical grid lines, the Gridded Distribution Rate (GDR) is an index of the goodness of fit of the grid. The Gridded Distribution Rate (GDR) is the proportion of all cells (n2) that are filled with at least one point (α) [81]: (20)

Next, we define the probability Pj which specifies the frequency of points for the j-th grid in the gridded Poincaré plot: (21) where b is the number of points in the grid [81]. And finally, we use the SE formula Eq (5) to compute the Gridded Distribution Entropy (GridEn) calculation: (22)

Illustration example: Gridded Distribution Entropy. Fig 6 illustrates 9 example fixations and associated GridEn. As we can see, the top row examples have high GridEn (see Fig 6A to 6C). Note that these are rough fixations with lots of irregular chaotic transitions and local minimums (or maximums). In the middle row, the trajectories are smoother without sharp unexpected transitions, but they are still somewhat complex with fewer marked peaks and valleys. For the bottom row, the inclusion of corrective saccades at the start of the plot in Fig 6G and the inclusion of part of a saccade following each fixation in Fig 6H and 6I are noteworthy. The saccade dominates the signal and sets the amplitude range for fixations to be large. Relative to this large range, the non-saccade associated signal has a very small range. This leads to a low GridEn entropy for these events.

thumbnail
Fig 6. The gridded Distribution Entropy.

In Fig 6A–6C there are three examples of fixation when this entropy is high. In Fig 6G–6I there are three examples of fixation when this entropy is low. In Fig 6D–6F there are three examples of fixations with entropy values intermediate between the high values and low values.

https://doi.org/10.1371/journal.pone.0291823.g006

GridEn is a feature that does not require phase space reconstruction. It utilizes the Poincare plot of the fixation trajectory to study the distribution of sample position across n separate regions. In the illustrative example (see Fig 7), the sample positions on each axis are scaled to a range of 0 to 1. The plot is then divided into nine square regions, and the number of points within each region is counted. A time series is considered complex when its points are highly distributed across all regions on the Poincare plot (see Fig 7A–7C). Typically, noisy fixation trajectories will be scored by a high value of GridEn. On the other hand, time series that incorporate fixation and corrective saccade will be scored by a low value of GridEn (see Fig 7G–7I). This is because the samples corresponding to the corrective saccade will appear as noticeably distinct points on the Poincare plot.

thumbnail
Fig 7. The Poincare plot for the GridEn for each fixation is illustrated in Fig 6.

Each Poincare plot is divided into a 3-by-3 grid (9 squares), corresponding to the fixation with the same letter in Fig 6. In the right corner of each square, we present the number of points in that region. Fig 7A–7C are examples of fixations that exhibit a high value of GridEn because points on the Poincare plot are highly distributed across all nine regions. When points on the Poincare plots are mostly distributed across 2 (or 3) dominant regions, the corresponding trajectories will be characterized by an average value of GridEn (see Fig 7D–7F). In Fig 7G–7I there are three examples of fixations in which points are mainly located in one dominant region. This produces a low GridEn.

https://doi.org/10.1371/journal.pone.0291823.g007

Metric 5: Phase entropy. The Phase Entropy (PhasEn) analysis [82] begins with a second-order difference plot (SODP) [83]. Let consider a data sequence X = x0, x1, x2, …, xn. From the given time series X = {xn}, we will denote the two sequences of differences as Y and Z: (23) (24)

The SODP is a plot of Y against Z. The slope from each line from the origin (0,0) to each point is determined. For this purpose, a four-quadrant arctan function is used to get the span of slope angles ∠ in the range (0, 2π): (25)

The scatter plot can be divided into a variable number of angular or radial sectors. In our case, we used the default value of k = 4. These 4 sectors have an angle span of 90 degrees ( radians) each. The parameter Mi where will denote the number of points in i-th sector. For each sector, we will calculate the cumulative slope Sθ: (26)

Next, we denote the cumulative slope of the entire plane as S, and divide each value of Sθ by it to obtain the probability distribution Pi for every sector: (27)

As a result, using the SE formula (5), we get the PhasEn: (28)

Illustration example: Phase entropy. In Fig 8, we illustrate 9 fixations with different levels of PhasEn. In Fig 9, we present SODP plots for each fixation in Fig 8.

thumbnail
Fig 8. The Phase Entropy.

In this plot, we present the 9 fixations. The top row consists of examples with high PhasEn, the bottom row consists of examples with low PhasEn and the middle row consists of examples with intermediate PhasEn.

https://doi.org/10.1371/journal.pone.0291823.g008

thumbnail
Fig 9. The Second Order Differences Plots (SODP) for each fixation illustrated in Fig 8.

The top row shows SODP plots for the 3 fixations previously illustrated with high PhasEn. The middle row displays 3 fixations with intermediate PhasEn, and the bottom row depicts 3 fixations with low PhasEn. For each fixation, the SODP is divided into 4 radial sectors. In the corner of each sector, we display the number of points contained within that sector. Fig 9A to.9C depict three examples of fixation where the SODP points are highly distributed across the four sectors. Fig 9D to.9F present three examples of fixation where the SODP points are less randomly distributed across the four sectors compared to the first three examples, but more randomly distributed than the last three examples. Finally, Fig 9G to.9I display three examples of fixation where the SODP points are predominantly located within a single dominant sector.

https://doi.org/10.1371/journal.pone.0291823.g009

The SODP plot can be divided into any number of regions but typically the default (which we use) is to divide the space into 4 quadrants (Fig 9). We can see that the first 3 fixations examples from Fig 9A to 9C begin with a large signal change in the rightward direction. These large signal changes produce points in the SODP clustered in the upper-right quadrant. All these fixations from Fig 9A to 9C are chaotic with a lot of transitions. After these initial large signal changes the changes in the remaining fixation points will tend to be clustered around the origin in the SODP plot, with some points in all sectors. And such dispersed distribution of points across sectors will produce a large Phase Entropy.

Now let’s look at the bottom row of Fig 8. Plots Fig 8G–8I exhibit a strong trend (drift) in one direction (left). This produces an SODP with values clustered in the lower-left (Fig 9). This uneven distribution of points over quadrants will produce a low Phase Entropy value. This also applies, to a smaller extent, to fixations in the middle row.

Metric 6: Spectral entropy. The Spectral Entropy (SpecEn) is calculated based on a Fast Fourier Transforms (FFT) of the signal [84]. Every motion is characterized by a countable number of frequency components. And generally, there are a few of the frequency components that are dominant.

Every frequency component νr of the signal X can be associated with the probability Pr such that: (29)

As a result, by using the SE formula (5), we can get the SpecEn in the next form: (30)

Illustration example: Spectral entropy. SpecEn is calculated based on a spectral density analysis of the eye-tracking signal. This analysis produces spectral density estimates for all possible frequencies for a given signal. The number of frequencies is a function of the length of the fixation. The spectral density estimates for all frequencies are normalized by the sum of the power spectral density across all frequencies. The Spectral Entropy is the result of computing the SE of these relative spectral density estimates.

In Fig 10 we present three fixations: one with high SpecEn Fig 10A, one with medium SpecEn Fig 10C and one with low SpecEn Fig 10E. The relative spectral density for each frequency is presented in Fig 10B–10F. Note that we have rescaled the ordinate to range from 0.0 to 0.001. This scaling allows us to visualize the majority of frequencies for which data are available. The fixation in Fig 10A appears relatively chaotic compared to those in Fig 10C and 10E. It produces a spectral density plot with many different relative density estimates which leads to a higher value of SpecEn. The fixation trajectory in Fig 10E is the smoothest, and therefore one would expect very low and consistent normalized spectral density in the higher frequencies. Since there are higher frequencies than low frequencies, the consistent normalized density estimates in Fig 10F result in a low estimate of SpecEn.

thumbnail
Fig 10. Explanatory plot for the Spectral Entropy.

The frequency plots have been scaled from 0 to 0.001 to better visualize the higher frequency estimates. In Fig 10A, we present a relatively chaotic fixation, and its corresponding frequency plot is shown in Fig 10B. The chart of normalized spectral density by frequency exhibits multiple levels, resulting in a high value of SpecEn. In Fig 10C, we present a fixation that is much less chaotic, leading to a narrower range of normalized density in the higher frequencies (refer to Fig 10D). In Fig 10E, we showcase a relatively smooth fixation, which is associated with very little variability in normalized density estimates in the upper frequencies (see Fig 10F).

https://doi.org/10.1371/journal.pone.0291823.g010

High-low analysis

To describe what kinds of fixations have very high and very low entropy measures, we have performed what we call a High-Low Analysis. We created a total of 36 figures by utilizing three datasets (GazeCom, Lund and OK Lab), six final entropy measures, and two figures per entropy measure (representing high entropy and low entropy, see Figs 11 and 12). The fixations are presented in the context of 30 samples before the fixation and 30 samples after the fixation.

thumbnail
Fig 11. GazeCom dataset: 36 horizontal fixations with the highest GridEn values.

The fixations are plotted in red. There are 30 signal samples before and after each fixation. The entropy value is displayed above each fixation chart. The classification of each fixation is shown in blue in the middle of each fixation chart. The fixation at the top left has the highest entropy value. Note the common occurrence of fixations that start too late on this page.

https://doi.org/10.1371/journal.pone.0291823.g011

thumbnail
Fig 12. GazeCom dataset: 36 fixations with the lowest GridEn values.

See the caption of Fig 11 for more details. The fixation with the lowest entropy value is shown in the bottom right. Note that 23 fixations start with a corrective saccade (CS) on this page.

https://doi.org/10.1371/journal.pone.0291823.g012

The three authors of this report each rated each fixation to decide if it was a “good” fixation or if it was a misclassified one (see Table 7). We considered six types of misclassifications as detailed in Table 6. We also considered if a fixation met one of three types of descriptions: (1) linear drift (LD), (2) a “bumpy” fixation (“BF”), or a noisy fixation (“N”). For examples of fixations with linear drift see the first two fixations in Fig 14. A “bumpy” fixation was a fixation with low-frequency noise. There are ten examples shown in Fig 20. A “noisy” fixation was a fixation with high-frequency noise. These fixations were most common in the OK Lab dataset (e.g., see Fig 21). This high-frequency noise may be ocular microtremor [85].

After each author rated the fixations, they met to reach a consensus on the rating of each fixation. The consensus ratings are presented in Figs 11 to 24.

thumbnail
Fig 13. GazeCom dataset: 36 fixations with the lowest PhasEn values.

See the caption of Fig 11 for more details. Note that 14 fixations exhibit linear drift (LD) on this page.

https://doi.org/10.1371/journal.pone.0291823.g013

thumbnail
Fig 14. GazeCom dataset: 36 fixations with high SampEn values.

See the caption of Fig 11 for more details.

https://doi.org/10.1371/journal.pone.0291823.g014

thumbnail
Fig 15. GazeCom dataset: 36 fixations with low SampEn values.

See the caption of Fig 11 for more details.

https://doi.org/10.1371/journal.pone.0291823.g015

thumbnail
Fig 16. GazeCom dataset: 36 fixations with low IncrEn values.

See the caption of Fig 11 for more details.

https://doi.org/10.1371/journal.pone.0291823.g016

thumbnail
Fig 17. GazeCom dataset: 36 fixations with high SpecEn values.

See the caption of Fig 11 for more details.

https://doi.org/10.1371/journal.pone.0291823.g017

thumbnail
Fig 18. GazeCom dataset: 36 fixations with high FuzzEn values.

See the caption of Fig 11 for more details.

https://doi.org/10.1371/journal.pone.0291823.g018

thumbnail
Fig 19. GazeCom dataset: 36 fixations with low FuzzEn values.

See the caption of Fig 11 for more details.

https://doi.org/10.1371/journal.pone.0291823.g019

thumbnail
Fig 20. Lund dataset: 36 fixations with low SpecEn values.

See the caption of Fig 11 for more details.

https://doi.org/10.1371/journal.pone.0291823.g020

thumbnail
Fig 21. OK lab dataset: 36 fixations with high GridEn values.

See the caption of Fig 11 for more details.

https://doi.org/10.1371/journal.pone.0291823.g021

thumbnail
Fig 22. OK lab dataset: 36 fixations with low SampEn values.

See the caption of Fig 11 for more details.

https://doi.org/10.1371/journal.pone.0291823.g022

thumbnail
Fig 23. OK lab dataset: 36 fixations with low SpecEn values.

See the caption of Fig 11 for more details.

https://doi.org/10.1371/journal.pone.0291823.g023

thumbnail
Fig 24. OK lab dataset: 36 fixations with high FuzzEn values.

See the caption of Fig 11 for more details.

https://doi.org/10.1371/journal.pone.0291823.g024

In the GazeCom dataset, there are 14 fixations with high FuzzEn, which include either a portion or the entirety of a saccade (see Fig 18). On the other hand, Low FuzzEn is associated with nine fixations that start too late (see Fig 19).

The table with all ratings of each fixation is available (see Data Availability Statement at end). The next step in the High-Low analysis was to identify ratings that occurred eight or more times on a single page. If a page had eight or more occurrences of the same rating, we considered that rating to be characteristic of the level of entropy (high or low) for a particular dataset. This is presented as Table 7.

Results

Table 7 summarizes the main results of this study. We found that high and low entropy values indicate different things for different datasets. In Table 7, we only include entropies where either high entropy, or low entropy, or both have some diagnostic value. For the GazeCom dataset, all six entropies were associated with some characterization of fixations. In the Lund dataset, only GridEn had any diagnostic value. In the OK Lab dataset, GridEn, SampEn, SpecEn, and FuzzEn had diagnostic value.

GazeCom dataset

A global examination of Table 7 indicates that all misclassifications of fixations occur in the GazeCom dataset. A common problem in the GazeCom dataset was that fixations started too late (code ST). Five of the twelve pages had eight or more fixations that started too late.

In the GazeCom dataset, high GridEn was associated with saccades that start too early (Table 7 and Fig 11). On the other hand, the low GridEn page had 23 fixations out of a total of 36 that started with corrective saccades.

In the GazeCom dataset, high PhaseEn is not associated with any particular misclassification or fixation description. However, a low PhaseEn is associated with a set of 14 fixations that display a linear drift (see Fig 13).

In the GazeCom dataset, on the high SampEn fixation page (see Fig 14), 10 fixations start too late. Eight of the fixations on the low SampEn page are contaminated with artifact (see Fig 15).

High IncrEn in the GazeCom dataset is not associated with any fixation issue. However, low IncrEn is associated with fixations that start too late and exhibit significant linear drift (see Fig 16).

High SpecEn in the GazeCom dataset is associated with fixations that start too late, fixations that end too early, and fixations affected by artifacts (see Fig 17). In this dataset, low SpecEn is not associated with any unusual fixations.

In the GazeCom dataset, there are 14 fixations with high FuzzEn, which include either a portion or the entirety of a saccade (see Fig 18). On the other hand, Low FuzzEn is associated with nine fixations that start too late (see Fig 19).

Lund dataset

As noted above, only a single entropy, SpecEn, had any diagnostic value for the Lund dataset. In this case, Low SpecEn produced 10 bumpy fixations (see Fig 20). High SpecEn was not diagnostic.

OK Lab dataset

As noted above, four of the six final entropies had some diagnostic value for the OK Lab Dataset. High levels of GridEn and FuzzEn indicated fixations with high-frequency noise (see Table 7, Figs 21 and 24). Low SpecEn also produces 10 noisy fixation examples (Fig 23). Low SampEn is associated with nine fixations with linear drift (Fig 22).

Discussion

The overall goal of this report was to evaluate what kind of information entropy metrics provide about eye movement fixations. We have evaluated this in 3 datasets: (1) GazeCom, (2) Lund, and (3) the OK Lab. The main result is that entropy metrics detect different fixation characteristics in each dataset. There are no conclusions regarding entropy measures that apply across datasets.

We took several steps to ensure that our evaluation of each dataset was based on comparable underlying data. The GazeCom dataset was sampled at 250 Hz, the Lund dataset was sampled at 500 Hz and the OK Lab dataset was sampled at 1000 Hz. We downsampled both the Lund dataset and the OK Lab dataset to 250 Hz to make them more comparable to the GazeCom dataset. We also equalized fixation durations across the datasets. But there are other characteristics of the three datasets that could not be equalized. The GazeCom subjects viewed video clips, the Lund subjects viewed static images and the OK Lab subjects were reading poetry. Also, the number of fixations differed widely across datasets. After duration equalization, the GazeCom dataset had 20,613 fixations, the Lund dataset had 145 fixations and the OK Lab dataset had 874 fixations. The 72 fixations we based our high-low analysis on corresponded to 0.35% of the GazeCom dataset, 49.7% of the Lund dataset, and 8.2% of the OK Lab dataset. We could have equalized these percentages by drawing random sub-samples from the GazeCom and OK Lab datasets. In one sense such an analysis might have been more fair. However, we reasoned that such an equalization would produce a less useful analysis of the information provided by entropy metrics.

The entropy metrics of fixations in the GazeCom dataset found a number of fixations that the present authors consider to be misclassified. The types of misclassifications we found in this dataset include fixations saccades that start too late, fixations that include an initial corrective saccade, fixations that include, or are influenced by signal artifact, fixations that include all or part of a saccade, and fixations that end too early. When interpreting this finding it is important to keep in mind that we are evaluating 0.35% of the fixations available in this dataset. On this basis, we cannot make any statement about the rates of misclassification in the GazeCom dataset as a whole. However, it is useful to review how classifications of eye movements were made in this dataset. Apparently, a large part of the classification work was performed by two novice annotators who were paid undergraduate students. Although eye movement experts were available and involved at several stages, the Lund dataset was classified by a single individual (Richard Andersson) who is an internationally known expert in eye movement classification. The OK Lab dataset was classified by an individual who had undergone very extensive training as outlined in [55]. Our entropy analysis of fixations in the GazeCom dataset might be useful for future users of these recordings. High and low entropy metrics might well be useful in an effort to improve the classification of fixation in the GazeCom dataset.

In addition, for GazeCom, two entropy measures (low PhasEn and low IncrEn) detected fixations that have marked linear drift. The movie-clip stimuli used for the GazeCom dataset did induce some smooth pursuit. Perhaps these entropy measures might assist in determining if some of the “fixations” detected with marked linear drift are actually periods of smooth pursuit.

For the Lund dataset, with far fewer fixations, only one entropy type, SpecEn, had any diagnostic value. Low SpecEn tended to find bumpy fixations, which have low-frequency oscillations. As far as we are aware, we are the first to distinguish such fixations, and we are not aware of prior research on the physiological basis or interpretation of such fixations. Although occasional bumpy fixations were noted in both the GazeCom and the OK Lab datasets, in no case were there eight or more bumpy fixations detected by any form of entropy.

In the case of the OK Lab dataset, three entropy types (high GridEn, low SpecEn, and high FuzzEn) detected “noisy” fixations, which are characterized by high-frequency oscillations. Although McCamy et al [85] state that ocular microtremor of fixation cannot be detected with the video-oculography (VOG) method, we think that this might not always be the case. The most likely interpretation of these noisy fixations is that they are examples of ocular microtremors that are detectable in some fixations from some individuals. The OK Lab data were collected on an EyeLink 1000, which is known to be a very accurate device with high precision [86].

Low SampEn detected fixations with linear drift in the OK Lab dataset. It is interesting to note that this is not the entropy that detected fixations with linear drift in the GazeCom dataset (low PhaseEn, low IncrEn). The recorded subjects in the OK Lab dataset were reading a poem. Such a stimulus does not induce smooth pursuit. These examples of linear drift in fixations might just be part of the normal variation in fixation stability. This also applies to the fixations with linear drift in the GazeCom dataset, although the latter did use stimuli that induced some periods of smooth pursuit.

All of these observations apply only to fixations that are longer than 40 msec. For GazeCom, we rejected 95 fixations or 0.26% of all fixations because they were too short. There were no fixations in either the Lund or the OK Lab datasets that were less than 40 msec. It is worth noting that Nystrőm and Holmqvist [87] set 40 msec as the minimum duration of fixation in their eye movement classification algorithm. Therefore, it appears that this limitation had a very limited impact on our analysis.

In summary, we applied six entropy metrics to the horizontal fixation signals from three different datasets (GazeCom, Lund, and OK Lab). The character of fixations identified by the entropy measures depended on the specific dataset. Therefore, it is not possible to draw a general conclusion about the significance of high or low entropy across these three datasets. In the GazeCom dataset, entropy measures detected certain misclassified fixations. However, given that we are only looking at a small portion of this dataset, we cannot make generalizations across the whole set of fixations. For the Lund dataset, low SpecEn detected fixations with low-frequency noise (“bumpy fixations”). The oculomotor basis of such fixations is not clear at this time. For the OK Lab dataset, it appears that high GridEn, low SpecEn, and high FuzzEn detected fixations with high-frequency noise. This high-frequency noise is likely to be an ocular microtremor that occurs in some subset of subjects and fixations.

References

  1. 1. Rigas I, Friedman L, Komogortsev O. Study of an Extensive Set of Eye Movement Features: Extraction Methods and Statistical Analysis. J Eye Mov Res. 2018;11(1). pmid:33828682
  2. 2. Lohr D, Berndt SH, Komogortsev O. An Implementation of Eye Movement-Driven Biometrics in Virtual Reality. In: Proceedings of the 2018 ACM Symposium on Eye Tracking Research & Applications. ETRA’18. New York, NY, USA: Association for Computing Machinery; 2018. p. 1–3. Available from: https://doi.org/10.1145/3204493.3208333.
  3. 3. Friedman L, Stern HS, Price LR, Komogortsev OV. Why Temporal Persistence of Biometric Features, as Assessed by the Intraclass Correlation Coefficient, Is So Valuable for Classification Performance. Sensors. 2020;20(16). pmid:32823860
  4. 4. Griffith HK, Komogortsev OV. Texture Feature Extraction From Free-Viewing Scan Paths Using Gabor Filters With Downsampling. In: ACM Symposium on Eye Tracking Research and Applications. ETRA’20 Adjunct. New York, NY, USA: Association for Computing Machinery; 2020. p. 1–3. Available from: https://doi.org/10.1145/3379157.3391423.
  5. 5. Friedman L, Nixon MS, Komogortsev OV. Method to assess the temporal persistence of potential biometric features: Application to oculomotor, gait, face and brain structure databases. PLOS ONE. 2017;12:1–42. pmid:28575030
  6. 6. Brasil A, Andrade J, Pinto L, Komati K. An Exploratory Study of Biometrics Using Eye Movement Trajectory Images Collected by Natural Image Stimuli. In: Anais do XLVII Seminário Integrado de Software e Hardware. Porto Alegre, RS, Brasil: SBC; 2020. p. 25–36. Available from: https://sol.sbc.org.br/index.php/semish/article/view/11314.
  7. 7. Brasil ARA, Andrade JO, Komati KS. Eye Movements Biometrics: A Bibliometric Analysis from 2004 to 2019. International Journal of Computer Applications. 2020;176(24):1–9.
  8. 8. Liebers J, Schneegass S. Gaze-Based Authentication in Virtual Reality. In: ACM Symposium on Eye Tracking Research and Applications. ETRA’20 Adjunct. New York, NY, USA: Association for Computing Machinery; 2020. p. 1–2. Available from: https://doi.org/10.1145/3379157.3391421.
  9. 9. Cheng S, Wang J, Zhang L, Wei Q. Motion Imagery-BCI Based on EEG and Eye Movement Data Fusion. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2020;28(12):2783–2793. pmid:33382658
  10. 10. Al Zaidawi SMK, Prinzler MHU, Schröder C, Zachmann G, Maneth S. Gender Classification of Prepubescent Children via Eye Movements with Reading Stimuli. In: Companion Publication of the 2020 International Conference on Multimodal Interaction. ICMI’20 Companion. New York, NY, USA: Association for Computing Machinery; 2020. p. 1–6. Available from: https://doi.org/10.1145/3395035.3425261.
  11. 11. Zargari Marandi R, Gazerani P. Aging and eye tracking: in the quest for objective biomarkers. Future Neurology. 2019;14:FNL33. https://doi.org/10.2217/fnl-2019-0012
  12. 12. Shelhamer M. Correlation dimension of optokinetic nystagmus as evidence of chaos in the oculomotor system. IEEE Transactions on Biomedical Engineering. 1992;39(12):1319–1321. pmid:1487297
  13. 13. Strogatz SH. Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry and Engineering. Westview Press; 2000.
  14. 14. West BJ. Fractal physiology and chaos in medicine. vol. 16. World Scientific; 2012.
  15. 15. Klonowski W. Signal and image analysis using chaos theory and fractal geometry. Machine graphics & vision. 2000; p. 403–431.
  16. 16. Eder J, Rembold H. Biosemiotics–a paradigm of biology. Biological signalling on the verge of deterministic chaos. Die Naturwissenschaften. 1992;79(2):60–67. pmid:1565162
  17. 17. Hilbert M. Information Theory for Human and Social Processes; 2020.
  18. 18. Byun S, Kim AY, Jang EH, Kim S, Choi KW, Yu HY, et al. Entropy analysis of heart rate variability and its application to recognize major depressive disorder: A pilot study. Technology and Health Care. 2019;27(S1):407–424. pmid:31045557
  19. 19. Keshmiri S. Entropy and the brain: An overview. Entropy. 2020;22(9):917. pmid:33286686
  20. 20. Wagner C, Nafz B, Persson P. Chaos in blood pressure control. Cardiovascular research. 1996;31(3):380–387. pmid:8681325
  21. 21. Zhang XD. Entropy for the complexity of physiological signal dynamics. Healthcare and Big Data Management. 2017; p. 39–53. pmid:29058215
  22. 22. Ando M, Nobukawa S, Kikuchi M, Takahashi T. Identification of electroencephalogram signals in Alzheimer’s disease by multifractal and multiscale entropy analysis. Frontiers in Neuroscience. 2021; p. 772. pmid:34262427
  23. 23. Cornforth DJ, Tarvainen MP, Jelinek HF. How to calculate Renyi entropy from heart rate variability, and why it matters for detecting cardiac autonomic neuropathy. Frontiers in bioengineering and biotechnology. 2014;2:34. pmid:25250311
  24. 24. Mammone N, Morabito FC. Enhanced automatic artifact detection based on independent component analysis and Renyi’s entropy. Neural Networks. 2008;21(7):1029–1040. pmid:18430547
  25. 25. Aştefănoaei C, Pretegiani E, Optican LM, Creangă D, Rufa A. EYE MOVEMENT RECORDING AND NONLINEAR DYNAMICS ANALYSIS—THE CASE OF SACCADES. Rom J Biophys. 2013;23(1-2):81–92. pmid:25698889
  26. 26. Aştefănoaei C, Creangă D, Pretegiani E, Optican LM, Rufa A. DYNAMICAL COMPLEXITY ANALYSIS OF SACCADIC EYE MOVEMENTS IN TWO DIFFERENT PSYCHOLOGICAL CONDITIONS. Rom Rep Phys. 2014;66(4):1038–1055. pmid:25698890
  27. 27. Veneri G, Federighi P, Rosini F, Federico A, Rufa A. Spike removal through multiscale wavelet and entropy analysis of ocular motor noise: A case study in patients with cerebellar disease. Journal of Neuroscience Methods. 2011;196(2):318–326. pmid:21262262
  28. 28. Shiferaw B, Downey L, Crewther D. A review of gaze entropy as a measure of visual scanning efficiency. Neuroscience & Biobehavioral Reviews. 2019;96:353–366. pmid:30621861
  29. 29. Shiferaw B, Crewther D, Downey LA. Gaze Entropy Measures Reveal Alcohol-Induced Visual Scanning Impairment During Ascending and Descending Phases of Intoxication. Journal of Studies on Alcohol and Drugs. 2019;80(2):236–244. pmid:31014469
  30. 30. Shiferaw BA, Crewther DP, Downey LA. Gaze entropy measures detect alcohol-induced driver impairment. Drug and Alcohol Dependence. 2019;204:107519. pmid:31479863
  31. 31. Harezlak K, Kasprowski P. Understanding Eye Movement Signal Characteristics Based on Their Dynamical and Fractal Features. Sensors. 2019;19(3). pmid:30717223
  32. 32. Harezlak K, Augustyn DR, Kasprowski P. An Analysis of Entropy-Based Eye Movement Events Detection. Entropy. 2019;21(2). pmid:33266823
  33. 33. Harezlak K, Kasprowski P. Application of Time-Scale Decomposition of Entropy for Eye Movement Analysis. Entropy. 2020;22(2). pmid:33285944
  34. 34. Startsev M, Zemblys R. Evaluating Eye Movement Event Detection: A Review of the State of the Art. Behav Res Methods. 2022;.
  35. 35. Harezlak K, Kasprowski P. Searching for Chaos Evidence in Eye Movement Signals. Entropy. 2018;20(1). pmid:33265121
  36. 36. Wang Q, Kim E, Chawarska K, Scassellati B, Zucker S, Shic F. On relationships between fixation identification algorithms and fractal box counting methods. In: Proceedings of the Symposium on Eye Tracking Research and Applications; 2014. p. 67–74.
  37. 37. Yokoyama H, Niwa Si, Itoh K, Mazuka R. Fractal property of eye movements in schizophrenia. Biological Cybernetics. 1996;75(2):137–140. pmid:8855352
  38. 38. Namazi H, Kulish VV, Akrami A. The analysis of the influence of fractal structure of stimuli on fractal dynamics in fixational eye movements and EEG signal. Scientific reports. 2016;6(1):1–7. pmid:27217194
  39. 39. Schmeisser ET, McDonough JM, Bond M, Hislop PD, Epstein AD. Fractal analysis of eye movements during reading. Optom Vis Sci. 2001;78(11):805–814. pmid:11763254
  40. 40. Gould JD. Pattern recognition and eye movement parameters. Perception & Psychophysics. 1967;2(9):399–407.
  41. 41. Gould JD, Dill AB. eye movement parameters and pattern discrimination. Perception & Psychophysics. 1969;6(5):311–320.
  42. 42. Yarbus AL. Eye movements and vision. Springer; 2013.
  43. 43. Noton D, Stark L. Scanpaths in eye movements during pattern perception. Science. 1971;171(3968):308–311. pmid:5538847
  44. 44. Noton D, Stark L. Scanpaths in saccadic eye movements while viewing and recognizing patterns. Vision research. 1971;11(9):929–IN8. pmid:5133265
  45. 45. Rizzo M, Hurtig R, Damasio AR. The role of scanpaths in facial recognition and learning. Annals of neurology. 1987;22(1):41–45. pmid:3631920
  46. 46. Anderson NC, Anderson F, Kingstone A, Bischof WF. A comparison of scanpath comparison methods. Behavior research methods. 2015;47(4):1377–1392. pmid:25540126
  47. 47. Shic F, Chawarska K, Bradshaw J, Scassellati B. Autism, eye-tracking, entropy. In: 2008 7th IEEE International Conference on Development and Learning. IEEE; 2008. p. 73–78.
  48. 48. Mohammadhasani N, Caprì T, Nucita A, Iannizzotto G, Fabio RA. Atypical visual scan path affects remembering in ADHD. Journal of the International Neuropsychological Society. 2020;26(6):557–566. pmid:31826774
  49. 49. Allsop J, Gray R. Flying under pressure: Effects of anxiety on attention and gaze behavior in aviation. Journal of Applied Research in Memory and Cognition. 2014;3(2):63–71.
  50. 50. Hooge I, Camps G. Scan path entropy and arrow plots: Capturing scanning behavior of multiple observers. Frontiers in psychology. 2013;4:996. pmid:24399993
  51. 51. Krejtz K, Duchowski A, Szmidt T, Krejtz I, González Perilli F, Pires A, et al. Gaze transition entropy. ACM Transactions on Applied Perception (TAP). 2015;13(1):1–20.
  52. 52. Dorr M, Martinetz T, Gegenfurtner KR, Barth E. Variability of eye movements when viewing dynamic natural scenes. J Vis. 2010;10(10):28. pmid:20884493
  53. 53. Startsev M, Agtzidis I, Dorr M. 1D CNN with BLSTM for automated classification of fixations, saccades, and smooth pursuits. Behav Res Methods. 2019;51(2):556–572. pmid:30411227
  54. 54. Larsson L., Nystrom M., and Stridh M. Detection of saccades and postsaccadic oscillations in the presence of smooth pursuit. IEEE Transactions on Biomedical Engineering, 2013:60(9), 2484–2493. pmid:23625350
  55. 55. Friedman L, Prokopenko V, Djanian S, Katrychuk D, Komogortsev OV. Factors affecting inter-rater agreement in human classification of eye movements: a comparison of three datasets. Behav Res Methods. 2023 Jan;55(1):417–427. pmid:35411475
  56. 56. Startsev M. and Agtzidis I. and Dorr M. Characterizing and automatically detecting smooth pursuit in a large-scale ground-truth data set of dynamic natural scenes. Journal of Vision 19.14 (2019): 10–10.
  57. 57. Rosner B. Percentage Points for a Generalized ESD Many-Outlier Procedure. Technometrics. 1983;25(2):165–172.
  58. 58. Zemblys R., Niehorster D. C., and Holmqvist K. gazeNet: End-to-end eye movement event detection with deep neural networks. Behavior Research Methods, 2019: 51(2), 840–864. pmid:30334148
  59. 59. Friedman Lee. Brief Communication: Three Errors and Two Problems in a Recent Paper: gazeNet: End-to-End Eye Movement Event Detection with Deep Neural Networks (Zemblys, Niehorster, and Holmqvist, 2019). Behavior Research Methods, 2020:52, no. 4, pp. 1671–80. pmid:32291731
  60. 60. Griffith H., Lohr D., Abdulin E. et al. and Komogortsev , Oleg V. GazeBase, a large-scale, multi-stimulus, longitudinal eye movement dataset. Sci Data 8, 184 (2021). pmid:34272404
  61. 61. Flood MW, Grimm B. EntropyHub: An open-source toolkit for entropic time series analysis. PLoS One. 2021;16(11):e0259448. pmid:34735497
  62. 62. Tomlin CJ, Axelrod JD. Biology by numbers: mathematical modelling in developmental biology. Nature reviews genetics. 2007;8(5):331–340. pmid:17440530
  63. 63. Bahill AT. Development, validation and sensitivity analysis of human eye movement models. Critical reviews in bioengineering. 1980;4:311–55. pmid:7016434
  64. 64. Enderle J. Models of Horizontal Eye Movements, Part I: Early Models of Saccades and Smooth Pursuit. vol. 5. Morgan & Claypool; 2010.
  65. 65. Kugiumtzis D. State space reconstruction parameters in the analysis of chaotic time series—the role of the time window length. Physica D: Nonlinear Phenomena. 1996;95(1):13–28.
  66. 66. Kember G, Fowler A. A correlation function for choosing time delays in phase portrait reconstructions. Physics letters A. 1993;179(2):72–80.
  67. 67. Aleksić Z. Estimating the embedding dimension. Physica D: Nonlinear Phenomena. 1991;52(2-3):362–368.
  68. 68. Gibson JF, Farmer JD, Casdagli M, Eubank S. An analytic approach to practical state space reconstruction. Physica D: Nonlinear Phenomena. 1992;57(1-2):1–30.
  69. 69. Wallot S, Mønster D. Calculation of average mutual information (AMI) and false-nearest neighbors (FNN) for the estimation of embedding parameters of multidimensional time series in matlab. Frontiers in psychology. 2018;9:1679. pmid:30250444
  70. 70. Rhodes C, Morari M. False Nearest Neighbors Algorithm and Noise Corrupted Time Series. Phys Rev E. 1997;55.
  71. 71. Bromiley P, Thacker N, Bouhova-Thacker E. Shannon entropy, Renyi entropy, and information. Statistics and Inf Series (2004-004). 2004; p. 1–8.
  72. 72. Porta A, Baselli G, Liberati D, Montano N, Cogliati C, Gnecchi-Ruscone T, et al. Measuring regularity by means of a corrected conditional entropy in sympathetic outflow. Biological cybernetics. 1998;78:71–8. pmid:9485587
  73. 73. Zadeh LA. Fuzzy sets. Information and Control. 1965;8(3):338–353.
  74. 74. Chen W, Wang Z, Xie H, Yu W. Characterization of Surface EMG Signal Based on Fuzzy Entropy. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2007;15(2):266–272. pmid:17601197
  75. 75. Liu X, Jiang A, Xu N, Xue J. Increment Entropy as a Measure of Complexity for Time Series. Entropy. 2016;18(1).
  76. 76. Delgado-Bonal A, Marshak A. Approximate Entropy and Sample Entropy: A Comprehensive Tutorial. Entropy. 2019;21(6). pmid:33267255
  77. 77. Pincus SM. Approximate entropy as a measure of system complexity. Proceedings of the National Academy of Sciences. 1991;88(6):2297–2301. pmid:11607165
  78. 78. Richman J, Moorman J. Physiological Time Series Analysis Using Approximate Entropy and Sample Entropy. American journal of physiology Heart and circulatory physiology. 2000;278:H2039–49. pmid:10843903
  79. 79. Friedman L, Komogortsev OV. Assessment of the Effectiveness of Seven Biometric Feature Normalization Techniques. IEEE Transactions on Information Forensics and Security. 2019;14(10):2528–2536.
  80. 80. Golińska AK. Poincaré plots in analysis of selected biomedical signals. Studies in logic, grammar and rhetoric. 2013;35(1):117–127.
  81. 81. Yan C, Li P, Liu C, Wang X, Yin C, Yao L. Novel gridded descriptors of poincaré plot for analyzing heartbeat interval time series. Computers in Biology and Medicine. 2019;109:280–289. pmid:31100581
  82. 82. Rohila A, Sharma A. Phase entropy: a new complexity measure for heart rate variability. Physiological Measurement. 2019;40. pmid:31574498
  83. 83. Thuraisingham RA, Tran Y, Boord P, Craig A. Analysis of eyes open, eye closed EEG signals using second-order difference plot. Medical & biological engineering & computing. 2007;45(12):1243–1249. pmid:17926075
  84. 84. Powell GE, Percival IC. A spectral entropy method for distinguishing regular and irregular motion of Hamiltonian systems. Journal of Physics A. 1979;12:2053–2071.
  85. 85. McCamy MB, Collins N, Otero-Millan J, Al-Kalbani M, Macknik SL, Coakley D, et al. Simultaneous recordings of ocular microtremor and microsaccades with a piezoelectric sensor and a video-oculography system. PeerJ. 2013 Feb 12;1:e14. pmid:23638348
  86. 86. Holmqvist, K. Common predictors of accuracy, precision and data loss in 12 eye-trackers. 2017, Unpublished. https://doi.org/10.13140/RG.2.2.16805.22246
  87. 87. Nystrőm M, Holmqvist K. An adaptive algorithm for fixation, saccade, and glissade detection in eyetracking data. Behav Res Methods. 2010 Feb;42(1):188–204. pmid:20160299