Fig 1.
Example of blink-locked pupillary response (BPR).
Raw pupil diameter time course example (top panel), and snapshots of recorded eye video (bottom panel). At the top panel, red bars indicate blinks, and blue vertical lines indicate the time of each eye image snapshot. At the bottom panel, the blue circle is a pupil area defined by the eye tracker, and the cyan circle is a corneal reflection. The snapshots are 4-s long and 2Hz frame rate. It clearly shows blink-locked pupillary response (BPR), that pupil constricts and re-dilates transiently after blinking. See the video (S1 Video) for vivid BPR examples.
Fig 2.
Experimental design and examination of factors affecting blink-locked pupillary responses.
(A) The fixation task was designed to measure BPR without the influences from other sources, such as saccade or other task-evoked components, as much as possible. In the experiment, four background-luminance levels were used. There was no stimulus, and luminance remained constant. Therefore, retinal illuminance decreases only when subjects blink. (B) Pupil and blink time series in an example run. The red dotted line indicates the blink onset or offset. Note that the pupil constricts and re-dilates transiently after blinks. (C–E) BPR time courses varying across several factors. Small vertical lines indicate the time-to-peak of each time course. (C) Grand average of BPR across subjects and background luminance. Line and shade indicate the mean and standard errors of the mean (SEM) across subjects and luminance, respectively. (D) BPR average across subjects within the background-luminance levels. BPR depended on the background luminance. The legend indicates the corneal flux density of the visual field (cd m–2 deg2). (E) BPR average across background-luminance levels within subjects. BPR shape and size were idiosyncratic across subjects. The lines were colored more reddish as their negative peak becomes larger. (F) BPR profiles are sorted into five quantiles by their peak amplitudes. Note that BPR amplitude is different blink-by-blink even when the effect of subject and luminance were ruled out. Also, note that the characteristic short-pipe shape of BPR maintains under any conditions.
Fig 3.
BPR is predicted to bias pupil size because blink rate and pattern vary across time, working-memory load, and subjects.
(A) Auditory oddball task structure. 0.2 s long high and low-tone beep sound presented every 2 s. Subjects are instructed to press one of the two buttons corresponding to low and high tones as soon and accurately as possible. The high-tone to low-tone presentation ratio is 2:8, 5:5, or 8:2, and the ratio is randomized across runs. (B) Blink rate mean and predicted BPR confound pattern in the auditory oddball task. The red horizontal bar and vertical lines indicate the presentation time of the auditory stimulation and the end of each trial, respectively. Note that BPR confound lasts till the subsequent trial. The pupil size decreased in the blink-occurred trial and recovered in the ensuing trial. (C) Delayed orientation estimation task structure. Bar (s) are presented 2 s long. It has four levels of difficulties to manipulate the degree of working-memory load (1,2,4 and 8 bars). Following a mask, subjects had to retrieve the memory to estimate the orientation of a post-cued bar. (D) Blink rate mean and predicted BPR confound pattern in the delayed orientation estimation task. The time 0 is the onset of the estimation epoch. The color indicates the number of bars to be memorized (N = 1,2,4 and 8 are denoted as black, blue, magenta, and red lines, respectively). Note that blink rate increased as the memory load increased, and thereby BPR is predicted to be biased to high memory load conditions. (E) Implicit models of pupillometry. They do not consider the effect of blink on pupil size at all (left), or treat the effect as a nuisance (right). (F) Realistic model of pupillometry. It considers blinks as confounders because blinks provide a backdoor from the cognitive-state variable to the pupil-size variable.
Fig 4.
Generative model-based decomposition of pupil-size measurements and example correction result.
(A) According to the realistic model, observed pupil size is an amalgam of response to cognitive state, BPR, and spontaneous fluctuation from other cognitive-independent sources. The goal is to decompose the pupil size as the sum of the components with a generative model and eliminate BPR specifically based on the estimated BPR component, making the data free from BPR confound. The algorithm estimates the BPR shape for each subject and BPR amplitude for each blink (e.g., sth subject’s jth blink is estimated here) (see Materials and Methods for the detailed description of the algorithm). (B) An example of BPR correction results. Note that the correction algorithm estimates the BPR amplitude for each blink (denoted as numbers above time course) based on the generative model. (C) Comparison between blink-locked time course from pre-BPR correction (black), post-BPR correction (green), and blink-free period (blue). Note that the characteristic short-pipe shape of BPR before correction is flattened and resembles a blink-free period after being corrected for BPR. Line and shade indicate the mean and standard errors of the mean (SEM), respectively. (D) Euclidean distance between the mean blink-locked time course and the mean blink-free time course in each subject. The distance decreases significantly after BPR correction (p < 0.001; ***).
Fig 5.
Temporal bias in blink patterns induces spurious distortion of pupil time course in the auditory oddball task.
(A) Schematic example addressing how BPR confounds pupillometry in the auditory oddball task. The pupil dilates by an auditory tone, which is the signal experimenters are interested in (blue). However, subjects spontaneously blink during the experiment, and it evokes BPR (red). Subjects tend to frequently blink about 500 ms after the stimulus (shown in Fig 3B). Therefore, BPR would distort the pupil response to the stimulus by decreasing it during the trial in which blink occurred and by increasing it during the subsequent trial. Linear interpolation of 3 s after a blink (orange) slightly reduced the pupil response in this case. (B) Simulation of pupil time course shape by BPR. BPR would induce significant and spurious distortion in shape. To identify the spurious distortion pattern by BPR clearly, two cases were selected. The cases where a blink occurred at the current trial (left panel) or right before the trial (middle panel) are shown (see Materials and Methods for details). The observed pupil time course (black) would be the sum of the pupil response to cognitive state (blue) and BPR (red). Note that it has increased and decreased by BPR (red) as shown in A. (right panel) In all trials, the increasing and decreasing effects are combined and happened such that predicted mean profile of pupil-size measurements (blue) so somewhat exaggerated the peak of the pupil dilation. (C) The spurious distortion in shape was identified as anticipated and fixed by the correction method. (Left panel) Pre-correction pupil time courses (black). When a blink occurred, the spurious decrease in the blink-occurred trials (dash-dotted line), the spurious increase in the post-blink trials (dashed line), and the spurious increase for the first 1 s and the following decrease in all trials (solid line) were observed as anticipated by the generative model. (Middle panel) Most of these spurious distortions were removed or attenuated after applying the model-based BPR correction method (green). (Right panel) The spurious distortions were over-corrected after applying the 3-s long linear interpolation correction method such that the all-trial average profiles of pupil-size measurements was substantially reduced (orange). Line and shade indicate the mean and standard errors of the mean (SEM) across subjects, respectively.
Fig 6.
Comparison of the blink-affected trials and the blink-free trials before and after correction.
(A) The time courses of pupil size before (left) and after correction (middle and right). The gray lines represent the mean time course of pupil-size measurements in the blink-free trials. The black line is the mean time course of pupil size in the blink-affected trials before correction. The green and orange lines represent the mean time courses of pupil size after correction with the model-based method and the linear interpolation method. Shades represent standard errors of the mean (SEM) across subjects. (B) The time courses of t statistics (“blink-affected”–“blink-free,” df = 23) of the difference between the blink-affected and the blink-free trials. Horizontal dashed lines indicate the values of t statistics where the significance level is 0.01, and t = 0.
Fig 7.
Contribution of BPR correction to the statistical power of the auditory oddball task.
(A) (top panels) Time courses of pupil size for different levels of sound-frequency oddity. Lines and shades correspond to the means and SEMs across subjects. (bottom panels) Time courses of t statistics (paired t-tests, df = 23) for the deviations of the high oddity (magenta) and low oddity (cyan) conditions from the medium oddity (blue) conditions. The left panel shows the results before correction; the middle and right panels show the results after being corrected using the model-based method and the interpolation method, respectively. (B-D) (left panels) Diagonal plot. Each marker indicates each condition pair, a subject has three markers each (high vs medium, medium vs low, and high vs low). Filled marker and error bars indicate the mean and SEM across subjects. Empty markers correspond to individual subjects, with different colors indicating different correction methods (green for the model-based method; orange for the interpolation method) and different symbols indicating different oddity pairs (see inset for detailed labels). (right panels) Boxplot. Corrected–uncorrected, which are equivalents to deviance from the diagonal line in the corresponding left panels. (Wilcoxon signed-rank test, (*, p < 0.05; **, p < 0.01; ***, p < 0.001)). The color scheme matched that shown in the left panels. (B) Mean difference in pupil dilation, (C) pooled SEM, (D) AUC between the conditions. (E) Comparison of the data uncorrected (black line), the data corrected with the model-based method (green), and the data corrected with the linear interpolation method (orange) in statistical power. The fraction of significant tests was plotted against the number of trials (see Methods and Materials for the detailed procedure).
Fig 8.
Temporal bias in blink patterns induces spurious distortion of pupil time course in the delayed orientation estimation task.
Pupil (solid line) and blink rate (dotted line) time course during the delayed orientation estimation task. Since BPR induces pupillary constriction, the pupil-size time courses corrected by the model-based method (green) and the linear interpolation method (orange) were larger than pre-correction (black) overall, but note that the linear interpolation method decreased the time course when it was a hill (upper blue arrow). Note that subjects tend to blink more often at the preparation epoch, and it induces a spurious dip at stimulus onset (red arrows). It was flattened after applying both of the correction methods. Line and shade indicate the mean and SEM, respectively.
Fig 9.
Contribution of BPR correction to the statistical power of the delayed orientation estimation task.
(A) (top panels) Pupil (solid line) and blink rate (dotted line) time course across the working-memory load conditions. The time 0 is the onset of the mask epoch. The pupil size increased as a function of working-memory load. The color indicates the number of bars to be memorized (N = 1,2,4 and 8 are denoted as black, blue, magenta, and red lines, respectively). Solid lines indicate the means across subjects, and the vertical dotted lines indicate the time span where the mean pupil of each trial is computed. Note that the more memory load is given, the higher the blink rate was, and thereby BPR would be more biased to the higher memory load conditions. The model-based method captured this bias so that the pupil-size time course under high memory loads increased more than low memory loads after the correction. (bottom panels) T values of the paired t-tests (df = 20) for the difference of each bar condition from the 2-bar condition. The left panel shows the results before correction; the middle and right panels show the results after being corrected using the model-based method and the interpolation method, respectively. (B-D) (left panels) Diagonal plot. Each marker indicates each condition pair, a subject has three markers each (1 bar vs 2 bars, 4 bars vs 2 bars, and 8 bars vs 2 bars). Filled marker and error bars indicate the mean and SEM across subjects. Empty markers correspond to individual subjects, with different colors indicating different correction methods (green for the model-based method; orange for the interpolation method) and different symbols indicating different working-memory load (the number of bars) pairs (see inset for detailed labels). (right panels) Boxplot. Corrected–uncorrected, which are equivalents to deviance from the diagonal line in the corresponding left panels (Wilcoxon signed-rank test, (*, p < 0.05; **, p < 0.01; ***, p < 0.001)). The color scheme matched that shown in the left panels. (B) Mean difference in pupil dilation, (C) pooled SEM, (D) AUC between the conditions. Panel B and C were zoomed in for clear visualization. See S2 Fig for zoomed-out plots. (E) Effects of the correction methods on statistical power in the delayed orientation estimation task. The number of trials required to reject the null hypothesis decreased by about 40% by the model-based method, whereas it increased 100% by the linear interpolation method.