Pupil Dilation Dynamics Track Attention to High-Level Information

It has long been thought that the eyes index the inner workings of the mind. Consistent with this intuition, empirical research has demonstrated that pupils dilate as a consequence of attentional effort. Recently, Smallwood et al. (2011) demonstrated that pupil dilations not only provide an index of overall attentional effort, but are time-locked to stimulus changes during attention (but not during mind-wandering). This finding suggests that pupil dilations afford a dynamic readout of conscious information processing. However, because stimulus onsets in their study involved shifts in luminance as well as information, they could not determine whether this coupling of stimulus and pupillary dynamics reflected attention to low-level (luminance) or high-level (information) changes. Here, we replicated the methodology and findings of Smallwood et al. (2011) while controlling for luminance changes. When presented with isoluminant digit sequences, participants' pupillary dilations were synchronized with stimulus onsets when attending, but not when mind-wandering. This replicates Smallwood et al. (2011) and clarifies their finding by demonstrating that stimulus-pupil coupling reflects online cognitive processing beyond sensory gain.


Introduction
Poets, philosophers, and artists throughout history have considered eyes to be portals to the mind. Science supports this intuition: eyes broadcast mind and animacy better than any other facial feature [1], with specific ocular cues linked to emotion (scleral size, [2]), information processing (eye-blinks, [3]), and intention (gaze direction, [4]). The pupil, especially, has been investigated as an index of mental states, given its tendency to dilate to salient stimuli such as emotionally arousing pictures [5], painful stimulation [6], and task-relevant numbers [7].
Pupil dilations due to changes in informational salience are much smaller than pupil dilations due to changes in luminance [8]. This difference in size reflects different biological pathways. Information-related dilations are associated with sympathetic activation of the superior sympathetic ganglion, and rarely exceed .5 mm [8,9]. In contrast, luminance-related dilations index parasympathetic activation of the Edinger-Westphal nucleus [10] and are generally between 2-4 mm [8,11]. For this reason, pupil dilations that index informational salience are only reliably measured under conditions of controlled light [7,[12][13][14].
Although previous studies have been careful to control luminance to reveal salience-related dilations, the standard analysis of pupillometric data is often temporally coarse: averaging pupillary diameter across a trial or using the maximum dilation during a pre-defined temporal window for analysis, e.g., [12,[14][15][16]. Although means and maxima are useful in discriminating responses between categories (e.g., emotional vs. neutral sounds, [14]; effortful vs. easy mathematical calculations, [17]), these metrics ignore temporal information that could further elucidate the dynamics of attention to information.
There is ample reason to believe that the pattern of pupillary dilation dynamics, when recorded at high temporal resolution, can provide a continuous index of attention. Pupil dilations have been shown to closely track norepinephrine (NE) release by the locus coeruleus (LC) [18,19] (see Figure 1). NE release in turn effects changes in individuals' attention and arousal by enhancing the synaptic responsivity of neurons to subsequent inputs [12]. As the LC is the sole source of NE-releasing fibers to the forebrain [18], and pupils can dilate with latencies as short as 200 ms [20], pupil size over time should provide an overt and objective measure of NE levels in the forebrain on a sub-second time-scale.
Previous studies have largely demonstrated the reliability of pupil diameter in acting as a reporter variable for LC activity in animal models, e.g. [19,21], or have focused on the relationship between baseline rates of LC activity, tonic pupil diameter, and arousal states [19,[22][23][24]. Less focus has been placed on understanding what insight the pupillary time-course can offer when it comes to real-time human information processing. Recently, Smallwood and colleagues probed whether the temporal pattern of pupil dilation indexes information processing under conditions of attention [25]. Their paradigm differed from other studies investigating the relationship between pupil size and attentional state, e.g. [12,26], in several key ways. First, their paradigm used the overall time-course of pupillary dilation as a dependent measure, rather than the peak dilation within a temporal window. Their design also allowed them to analyze only the time-periods during which participants viewed stimuli that did not require a response, avoiding potential confounds associated with motor planning and execution. In this way, they tested not only whether phasic dilations occurred during periods of high attention (which has been well-established in the literature: [27][28][29]), but also whether the specific timing of these dilations could be used to discriminate between conditions of task focus versus mind-wandering. The LC-NE system is associated with ''alertness'' for incoming external stimuli (exogenous attention [30,31]), with sustained endogenous attention generally considered the responsibility of cholinergic pathways [30,32,33]. Thus, Smallwood and colleagues hypothesized that when participants were motivated to attend to exogenous cues, their pupil dilations would couple to the presentation of exogenous cues (''task-evoked dilations'') compared to when they were not motivated to attend to those cues [25]. Their results supported their hypothesis.
Consistent with their data, Smallwood and colleagues concluded that mind-wandering essentially ''de-couples'' participants from external stimuli. Further supporting this conclusion, recent research found that pupil dilations systematically varied with spontaneous fluctuations of engagement [34]. Neither study, however, makes strong claims as to what about the stimuli causes pupil dilation coupling under task focus, possibly because there are two viable explanations: (1) changes in low-level cues that are attended more in the task-focus condition (in Smallwood and colleagues' paradigm, these were luminance changes between foreand background colors with the disappearance of the fixation cross and appearance of the next stimulus), and (2) changes in (high-level) information that necessarily accompanied each new stimulus. Here, we tested whether pupil dilations reflect attention in the absence of luminance changes on the retina. If so, this would rule out the first (luminance) explanation as the primary cause of Smallwood and colleagues' 2011 results, leaving the striking explanation that the pupil dilation time-course can index attention to high-level informational changes [25]. To do so, we directly replicated Smallwood and colleagues' paradigm while controlling for luminance changes.

Ethics Statement
This research was approved by Dartmouth College's Committee for the Protection of Human Subjects, Protocol #20951. Written informed consent was obtained from all participants at the start of the study. Only individuals with normal vision, or vision corrected-tonormal using contact lenses, were allowed to participate in the study. Individuals wearing glasses were not used due to the propensity of some glasses to occlude the pupillary response. 16  [19], and shows the close positive association between baseline locus coeruleus activity and tonic pupil diameter. Rajkowski and colleagues recorded from a single LC neuron in the monkey, and used a remote eye-tracking camera to measure pupil dilation during a target detection task. doi:10.1371/journal.pone.0102463.g001 participants (11 females) passed quality control cut-offs (see below); data from these participants is discussed below.

Eye-tracking and Quality Control
Pupil diameter was collected from the left eye at 120 Hz using the ASL Eye-Trac 6 eye-tracker (Applied Sciences Laboratories, Bedford, MA). Any missing values (e.g., due to blinks or machine artifact) were interpolated using basic linear interpolation. In accordance with quality control measures instituted by Smallwood and colleagues [25], individuals whose data required in excess of 40% interpolation, and/or whose responses showed below-chance accuracy during experimental trials were rejected from analysis. Resultant pupil data was median filtered (order 5), low-pass filtered (cutoff frequency 10 Hz), and z-scored to account for individual differences in pupil size. In order to maintain any task-related differences in pupillary response, z-scoring was completed across both tasks per individual.

Design and Procedure
Design for CR and WM tasks replicated Smallwood et al. [25] (see Figure 2). In both tasks, strings of numbers interleaved with fixation crosses were shown to participants. Each number was presented for 1000 ms; fixation crosses appeared onscreen for between 900-2300 ms. Response trials occurred every 2-5 numbers, and were indicated by a colored probe. In the CR task, this response probe was a number, and participants were asked to indicate whether that number was even or odd. In the WM task, the probe was a colored question mark, and participants performed a 1-back task, indicating the parity of the preceding number. Importantly, the CR task did not require attention to non-probe trials, whereas the WM task demanded that participants attend to and remember each non-probe number. Each task was 10 minutes in duration, and contained 48 probes.
Smallwood and colleagues utilized response probes that were colored either red or green, on a white background. Non-probe numbers were black. Given pupillary sensitivity to luminance [35], the current experiment used an isoluminant paradigm. This allowed us to test whether the pupils dynamically coupled to attended stimuli in the absence of luminance changes. To determine the RGB values perceived as isoluminant by each individual, participants first completed a short color task, coded in Matlab and presented on a Cathode Ray Tube (CRT) monitor. A CRT monitor was used because of its more sensitive refresh rate. In this task, participants saw a small box colored red, green, or blue on a gray background. For any value that was not isoluminant with the gray background, this box would appear to flicker. Participants used the keyboard arrow keys to adjust the luminance of these boxes, and save the values at which flickering ceased for each color (i.e., at which isoluminance was reached). The corresponding RGB values were then used to tailor participant-specific CR and WM tasks, presented using E-Prime 2.0 Professional software on the CRT monitor. In this paradigm, participants viewed strings of green non-probes and fixation crosses on a red background, and responded when blue probes appeared (see Figure 2). To control for eye movements, participants were instructed to keep their gaze on the fixation cross and stimuli that appeared in the center of the computer screen throughout the duration of the experiment. Onscreen luminance remained constant, with only stimulus salience differing across tasks. The order in which these tasks were completed was counterbalanced across subjects. Participants sat approximately 30 inches from the eye-tracker. A chin rest was utilized in order to keep participants stationary during the task.

Results
Following the procedure outlined in Smallwood et al. [25], analyses were constrained to participants' pupillary behavior in response to presentation of non-probes, as only these differed in salience across conditions. Pupillary time-series data were locked to the 2.5 seconds following presentation of each of the 169 nonprobes for each participant, for each task. These time-courses were normalized across both tasks to preserve their responses across CR and WM conditions. Time-courses were then averaged into  Figure 3a). These bins corresponded to the time period occurring 1-1.75 seconds following onset of the non-probe stimulus. This is consistent with findings that pupillary dilations in response to cognitive tasks peak approximately 1 second after stimulus onset [36], and suggest that task-evoked responses were present only in the WM condition. There was no significant difference in the number of blinks participants made (t(15) = 2.278, p = ns) or their percent accuracy (t(15) = 2.901, p = ns) across WM and CR tasks. We also found no significant difference in tonic pupil size prior to probes as a function of accuracy (t(15) = 2.126, p = ns) within task.
We also deconvolved the pupillary responses associated with each condition, using the automated pupil dilation deconvolution method recently developed by Wierda and colleagues [36]. Each participant's normalized mean response for the 2.5 seconds following stimulus onset was downsampled to 40 Hz, and averaged into 99 25 ms bins. The algorithm deconvolved these responses by calculating the misfit between the observed and predicted pupil dilation patterns, based on the pupillary response function of Hoeks and Levelt [37], and calculated the average strength of the attentional pulse associated with both conditions. We compared these attentional strengths using a paired t-test. The attentional pulse associated with non-probes in the WM condition (M1 = .254, SD1 = .28) was significantly stronger than the pulse associated with CR non-probes (M2 = .064, SD2 = .11), t(15) = 2.333, p = .03. The deconvolved pupillary responses associated with WM and CR conditions are shown in Figure 4.

Discussion
Five decades of pupillometric research left the tantalizing impression that pupillary dynamics offer a real-time window on attention. Recently, Smallwood and colleagues [25] came closest to achieving this possibility. Here, we extend their results by controlling for low-level (luminance) changes. We replicated Smallwood and colleagues' finding that the dynamic pupillary response discriminates trials during which participants are attending versus not attending exogenous cues. That is, pupillary dilations were time-locked to the appearance of exogenous cues only when those cues were attended, and this effect did not depend on low-level luminance changes accompanying those cues.
The current findings suggest that the dynamics of pupil dilation comprise a high-temporal resolution measure of attention. Here, we have shown that pupil dilations are capable of indexing information changes independent of low-level visual changes (luminance). The objectivity of pupillometry and the relative ease with which it can be obtained, renders it an ideal metric for a multitude of research questions. Although the Smallwood et al. paradigm [25] uses pupil dilation dynamics to track the changing salience of discrete visual stimuli, we see no reason why these pupillary dynamics could not be used to index moment-bymoment fluctuations of attention to other input modalities (e.g., audition and touch). While the concept of using pupil dilation dynamics as an index of cognitive processing is not novel [17], it remains underutilized. Recent technological advances now make it possible to achieve the temporal resolution necessary to measure real-time fluctuations in attention -not just to changes in light but to changes in information. Pupil dilation dynamics thus afford a temporally-sensitive portal onto the processing of another mind.  [36] was used to derive the attentional pulse strengths associated with non-probe stimuli in WM and CR conditions. This attentional pulse was significantly stronger in the WM condition (M1 = .254, SD1 = .28) than in the CR condition (M2 = .064, SD2 = .11), t(15) = 2.333, p = .03. The deconvolved pupillary responses to non-probes are displayed above, with error bars indicating one standard error of the mean. doi:10.1371/journal.pone.0102463.g004