Decoding Vigilance with NIRS

Sustained, long-term cognitive workload is associated with variations and decrements in performance. Such fluctuations in vigilance can be a risk factor especially during dangerous attention demanding activities. Functional MRI studies have shown that attentional performance is correlated with BOLD-signals, especially in parietal and prefrontal cortical regions. An interesting question is whether these BOLD-signals could be measured in real-world scenarios, say to warn in a dangerous workplace whenever a subjects' vigilance is low. Because fMRI lacks the mobility needed for such applications, we tested whether the monitoring of vigilance might be possible using Near-Infrared Spectroscopy (NIRS). NIRS is a highly mobile technique that measures hemodynamics in the surface of the brain. We demonstrate that non-invasive NIRS signals correlate with vigilance. These signals carry enough information to decode subjects' reaction times at a single trial level.


Introduction
The ability of organisms to maintain their focus of attention and to remain alert to stimuli over prolonged periods of time is termed ''vigilance'' [1]. For example, during long-lasting activities such as steering a car for hours the speed and accuracy of responses to incoming stimuli like traffic signs and other cars vary over time. Obviously, in real-world scenarios any attention failures can involve high risks. So it would be useful to have a device that measures neural correlates of attentional performance in real time and in the real world. Such a device could warn when vigilance is low or even induce events that increase attention [2]. It has long been known that the neural processes underlying attentional performance can be measured with neuroimaging techniques [3][4][5].
Several studies have demonstrated the suitability of EEG and fMRI for real-world monitoring of mental states and for braincomputer-interface (BCI) applications [6][7][8][9][10][11][12][13][14][15][16]. EEG has the potential to measure fast electrical signals from the surface of the scalp. However, for attention monitoring EEG signals might not be the first choice. EEG is very sensitive to motion artifacts [17] and the preparation is tedious, although recent developments might in future decrease preparation time drastically. Furthermore, an EEG system can always be influenced by other electronic or magnetic devices, which complicate its usability in technical environments like (car or airplane) cockpits [but see 18 for an application during simulated driving]. Finally, EEG-based BCIs are very useful when high bit-rates are necessary (for example communication devices). However, for monitoring slow fluctua-tions of attention and vigilance such high bit-rates might not be necessary.
A different approach would be to use functional magnetic resonance imaging (fMRI). Numerous studies have demonstrated correlates of attention in BOLD signals as measured with fMRI, ranging from sensory regions to parietal and prefrontal cortex [3,5]. However, functional magnetic resonance imaging is stationary and thus lacks the mobility needed for natural environmental settings.
Here, we assessed the possibility of attention monitoring using Near-Infrared Spectroscopy (NIRS) as a potentially suitable technique for measuring brain activity in natural environments. Similar to fMRI, NIRS measures the blood oxygenation level of the superficial layers from the surface of the human brain and can distinguish between concentration changes of oxygenated and deoxygenated hemoglobin (HbO and HbR). Thereby, it measures an effect comparable to the blood oxygenation level dependent (BOLD) effect in fMRI [19,20]. While concentration of HbO is expected to increase after focal activation of the cortex due to higher blood flow, HbR is washed out and decreases [21,22]. So NIRS might potentially be able to register attention-related BOLD-signals near the surface of the brain, particularly in parietal and prefrontal cortex. Although NIRS has the substantial benefits of low-costs, easiness to handle and, especially, its mobility, it suffers from spatial limitations as compared to fMRI. While the spatial resolution can be enhanced to a sub-centimeter space [23][24][25][26], the depth in which hemodynamics can be measured is physically limited to the upper layer of the cortex. Thereby, fMRI remains the 'gold standard' in localizing hemodynamics.
NIRS has been successfully used to investigate the neural signatures of performing vigilance tasks, typically by contrasting a task with a resting or control condition [27][28][29][30][31][32][33]. It has been shown that typically the involvment of the right hemisphere is stonger relative to the left hemisphere in vigilance tasks [27,28,31], however the involvment of both hemispheres also depends on the vigilance task used [28,31]. Interestingly NIRS has been used in ecologically valid environments to investigate vigilance [32]. However, the goal in these previous studies was to identify regions that show differences in brain activity between performing a vigilance task compared to some kind of baseline condition, for example an easy control task or rest. Importantly, a region that is active during the subjects' involvement in a task that requires sustained attention, does not necessarily also reflect performance fluctuations during the execution of the task. It could be possible that depending on the chosen vigilance task, a brain region shows an increased response that is related to specific properties of the task that is not necessarily related to vigilance, for example difficulty. Therefore, in the present study we are interested in measuring and relating fluctuations in the NIRS signal to performance fluctuations during the performance of a task that requires sustained attention. With this approach we can directly test, whether activity in a specific brain region reflects a potential measure for vigilance that is not confounded with differences between the vigilance and a control task (for example difficulty). Furthermore, our approach is closer to realistic scenarios in which it would be necessary to monitor performance fluctuations during vigilance tasks and not simply detect whether a monitored subject . ( ) is performing a vigilance or some other kind of task. Specifically, in the present study we want to investigate whether NIRS signals are systematically modulated by vigilance fluctuations (as measured by changes in response times of a task) (Analysis I). Furthermore, we want to use the spatial-temporal pattern of the recorded NIRSsignal to decode single trial reaction times (Analysis II). Another neuroimaging technique that could potentially be used to associate brain signals with performance is Transcranial Doppler Sonograply (TCD). TCD has already been shown to be sensitive to critical event detection during a vigilance task [34]. TCD measures global blood flow changes while NIRS is a more focal measure of cortical hemodynamics.
NIRS is also to some degree feasible for BCI [35][36][37]. NIRS is relatively motion artifact insensitive; it is non-invasive, portable and economic [38]. Although the hemodynamic response measured with NIRS is a slow response, also the attentional state fluctuates slowly. Thus, the time resolution of NIRS could be sufficiently high to acquire neural correlates of vigilance from the surface of the brain.

Materials and Methods
Subjects 9 healthy subjects (22-25 years old; 4 female) without any history of neurological or psychiatric disorders participated voluntarily in our experiment and received financial compensation. All of the participants had normal or corrected to normal vision and were right handed as indexed by self-report. One subject had to be excluded from the analysis because of technical problems during the measurement. The study was approved by the Ethics Committee of the Universitä tsklinikum Leipzig and all participants gave their written consent prior to the experiment.

Experiment
The experimental paradigm used to measure changes in vigilance was chosen to be very similar to one that was presented in a previous fMRI study on attentional fluctuations [5]. This task was chosen because in this fMRI experiment slips of attention could then be compared to changes in the BOLD response [5]. FNIRS measures a similar signal like fMRI, however limited to the surface of the brain, therefore we expected to find similar modulations in the fNIRS signal. Subjects executed a global-local Stroop paradigm. The visual stimuli consisted of multiple small letters (either S or H) that were arranged so that the shape of either a large S or a large H was formed (see Figure 1). Half of the trials were congruent, i.e. the global and local letters were mapped to the same response, and the other half was incongruent. Before the start of each block subjects were instructed verbally to attend either to the local or the global letters. In the first 3 blocks subjects were instructed to attend to either the global or the local letters. Half of the subjects were instructed to attend to the global letters in the first 3 blocks and to the local letters in the subsequent 3 blocks. The other subjects were instructed to attend to the local letters in the first 3 blocks and to the global letters in the subsequent 3 blocks. Stimuli had a size of 666108 pixel and were presented for 200 ms on a 150 display with a resolution of 10246768 pixel. After visual presentation subjects had to indicate the identity of the attended letter by button press. Inter-trial intervals (ITI) between visual presentations had a mean duration of 7.9 s (ranging from 7 to 12 s with an exponential decay and resulting in a duration of 474 s for one block). Each subject performed 6 blocks of the experiment during which 60 trials (30 congruent, 30 incongruent) were presented. Taken together the duration of the whole experiment (without short breaks between the blocks) was 47 m and 24 s. The subject's single trial performance (i.e. mean corrected reaction time) was used as an indicator for the current vigilance state for the further analyses. Please note, that the experiment was not aimed to investigate conflict processing during a Stroop task. Importantly, here we used the Stroop paradigm as an example of a task that requires relatively high levels of attention to perform it correctly. Therefore, we corrected for the behavioral effects, i.e. mainly reaction time differences between congruent and incongruent trials by subtracting the condition specific mean reaction times. Furthermore, the Stroop task is a non-typical vigilance task that is, in the here presented used form, more similar to a psychomotor vigilance task [39]. Typical vigilance tasks involve a relatively rare event that has to be detected between very frequent events. Both the frequent and the rare event itself are very easy to detect [40,41]. However, in a classical vigilance task the subject is asked to report only occurrences of the rare event. During such typical vigilance tasks a vigilance decrement is reported [42], i.e. increased reaction times and increased error rates over time. The Stroop task used here is useful for the acquisition of relatively high sampled measure of vigilance (reaction times) during a task that requires attention. However, because it is very different to typical vigilance tasks effects such as the vigilance decrement might not be observable.

NIRS acquisition and preprocessing
During the experiment, blood oxygenation at the surface of the subjects' brain was measured with a NIRS system, which consisted of 16 detectors and 16 emitters (NIRScout 16-16, NIRx Medizintechnik GmbH, Berlin, Germany) at two wavelengths (850 and 760 nm). Based on previous findings [5], we chose fiber optode positions to cover the frontal and parietal areas of the subject's head, providing a total of 44 useful channels where source and detector were at a distance of between 2 and 3 cm from each other. This arrangement is shown in Figure 2. To guarantee optimal accuracy in optode localization and convenience for the subjects, the emitters and detectors were integrated into a commercially available EEG cap (www.easycap.de) with 128 possible positions. NIRS data were continuously sampled with 6.25 Hz.
For visual inspection and demonstration, we employed the freeware MATLAB toolbox NFRI (www.jichi.ac.jp/brainlab/ tools.html) by Singh and Dan [43], which takes EEG 10-20 positions, given by the EEG cap used, as references to estimate brain regions underlying the NIRS channel locations. This toolbox thereby enables statistical results for each channel to be plotted on the surface of a schematic brain as used in Figure 2. To evaluate To evaluate the cortical structures underlying the NIRS channels the MNI coordinates were transformed to Talairach space [44,45] and looked up in a brain atlas [46]. doi:10.1371/journal.pone.0101729.t001 the cortical structures underlying the NIRS channels we transformed the MNI coordinates to Talairach space [44,45] and looked them up in a brain atlas provided by [46] (see Table 1). The channels covered brain regions that have been reported to be associated with lapses of attention [5] such as the middle frontal gyrus, precuneus, and superior parietal lobe. However, with NIRS it is only possible to measure BOLD responses on the surface of these brain regions. Raw data was filtered at 0.25 Hz using a third order digital Butterworth low-pass filter. The cutoff frequency was chosen to attenuate high frequency noise and the cardiovascular signal (i.e., breathing and heartbeat). Further analysis with different cutoff frequencies (Figure 3) revealed that potential noise in the frequency of breathing (0.2-0.3 Hz) is not affecting the results of the performed decoding analysis. Attenuation changes of both measured wavelengths were transformed to concentration changes of oxy-and deoxygenated hemoglobin (HbO and HbR) using a modified Beer-Lambert law [47].

Behavioral Data
Mean response times as well as hit rates for congruent and incongruent trials were calculated. We also calculated hit rates and response times for the six runs and for the two attention conditions (attend local or attend global) individually. For all further analysis only reaction times of correct trials were considered. In order to remove condition specific reaction time differences, we calculated the average reaction time for the congruent and for the incongruent trials for each run of each subject. Then these mean reaction times for the congruent and incongruent trials were subtracted from the individual congruent and incongruent trials respectively. Please note that after this correction it is impossible to infer from the reaction time alone, whether a trial was congruent or incongruent. However, more importantly, the corrected reaction times reflect relative performance fluctuations during the execution of the task independent of potential congruency effects.

Analysis I: Finite Impulse Response Analysis
The preprocessed NIRS-data was analyzed using custom software. In a first analysis we implemented a general linear model (GLM) that makes no assumptions about the shape of the hemodynamic response (e.g. the shape of the hemodynamic response function). Therefore 12 consecutive stick function regressors were estimated to model the event-related hemodynamic response (ER-HR). The data were recorded with a sampling frequency of 6.25 Hz, i. e. much faster than in fMRI paradigms (typically around 0.5 Hz). Therefore, in contrast to finite impulse response (FIR) analysis for fMRI data, we modeled every 6 th time point with a stick function instead of impulses for each time point. Furthermore we included 12 parametric regressors [48], one for each of the 12 onset regressors which modeled the modulation of the hemodynamic response due to variations in the mean corrected response times, resulting in a total of 24 regressors. We further refer to this set of regressors as event-related parametric modulated (ER-PM). The model used in the GLM furthermore included discrete cosine functions to attenuate slow drifts below 1/128 Hz.
The regression approach described above is similar to the analysis used in the study from Weissman [5]. Analysis II: Decoding The aim of Analysis II was to predict (mean corrected) single trial behavioral response times using support vector regression (SVR). For the prediction we used all 44 channels from both chromophores of the hemoglobin (HbO and HbR) resulting in 88 channels. Further preprocessing steps of the data after the application of the modified Beer-Lambert law included linear drift correction and z-normalization. Both preprocessing steps were performed separately for each run. Data used for the decoding was extracted by a sliding window that moved with a step size of 3 samples and had a width of 7 samples. The range covered by the sliding window analysis started at 25 s (231 samples) and ended at 14.7 s (92 samples) relative to the stimulus onsets. For each trial the data from all the 88 channels x 7 time points were transformed into a vector and used as features for space-time decoding [49]. The advantage of this approach is that spatial patterns as well as temporal patterns (for example temporal gradients) can enhance the decoding performance. Correct trials from 5 of the 6 experimental runs were used to train the SVR. Trials from the run that was not used for training were used to estimate the accuracy of the prediction by calculating the correlation coefficient between predicted and actual (mean For illustration purposes the FIR-parameters for an HbO-channel in parietal (left) and prefrontal (right) areas are plotted. A) Time courses of FIR-parameter estimates for 2 reliable (based on a t-test on the parameter estimates) HbO-channels in parietal (left) and prefrontal (right) areas (the selected channels are marked with black circles in B below). Red: FIR parameter estimates of the ER-HR; Green: FIR parameter estimates of the ER-PM. The ER-HR can be thought of the mean averaged response to the stimuli and the ER-PM is the reaction time dependent modulation of the amplitude. For the two channels this means that for long reaction times the amplitude of early time points will be reduced and for late time points the amplitude will be increased. Please note the difference in amplitude between the prefrontal and parietal channel. B) FIR results of event-related hemodynamic response (ER-HR, top) and event-related parametric modulation (ER-PM, bottom) for the most significant time points (6 and 7 s after stimulus onset for the ER-HR and ER-PM, respectively) plotted on a 3D head surface. Black circles indicate the selected channels for which the time course is plotted in A. doi:10.1371/journal.pone.0101729.g005 corrected) reaction times. Each experimental run was used once as test data set, i.e. we implemented a 6-fold cross validation. The correlation coefficients were Fisher-Z normalized and averaged for each subject across the cross validation steps. The accuracy (i.e. mean Fisher-Z corrected correlation) was then tested for statistical significance across subjects for each step of the sliding window.

Analysis III: Time-frequency decoding
The same analysis as described above was also performed for individual frequencies of the NIRS data. We first calculated a wavelet transformation on the signal and separated the signal between 0.001 and 0.25 Hz into 250 frequency bands. Afterwards we performed a decoding analysis very similar to the one described above, but now for each derived frequency band separately. Instead of space-time decoding, the signal within the sliding window was averaged. Figure 4 shows the behavioral results. The performance (correct responses) and the response times (only for the correct responses) of the congruent trials were compared to the incongruent trials. Through the whole experiment and combined across the congruent and incongruent condition there were 10.5 errors (standard error of the mean (SEM): 2.16) ranging from a maximum of 19 to a minimum of 4 errors. In the congruent condition participants made 2.75 errors (SEM: 1.03) ranging from a maximum of 9 to a minimum of 0 errors. In the incongruent condition participants made 7.75 errors (SEM: 1.03) ranging from a maximum of 18 to a minimum of 3 errors. There were significant effects in both measures (performance: t(7) = 2.673 p, 0.05, g = 1.17, response time: t(7) = 26.016 p,0.001, g = 20.79). We calculated a 262 repeated measurement ANOVA on the performance with the factors congruency (congruent, incongruent) and attention (global, local) that revealed a significant main effect for congruency (F(1,7) = 7.14 p,0.05) but not for attention (F(1,7) = 0.796 p = 0.402). The interaction congruency X attention was also not significant (F(1,7) = 0.092 p = 0.771). The same 262 repeated measurement ANOVA on the response times revealed also a significant main effect for congruency (F(1,7) = 36.198 p, 0.001) but not for attention (F(1,7) = 0.737 p = 0.419) and not for the interaction (F(1,7) = 0.847 p = 0.388). Therefore, we concluded that the behavior was similar between the two attention conditions.

Behavior
A 266 repeated measurement ANOVA with the factors congruency (congruent, incongruent) and time (run 1, 2, 3, 4, 5, and 6) was calculated on the response times to test for time sequence effects. The ANOVA again revealed a significant main effect for congruency (F(1,7) = 36.198 p,0.001) and additionally for time (F(5,35) = 3.846 p = 0.007). The interaction congruency X time was also significant (F(5,35) = 4.692 p = 0.002). Please note that after the first three runs subjects had to switch their attention from either global to local or vice versa, which potentially caused the strong response time increase in run 4.
In a final analysis we tested for sequence effects that are dependent on the global/local-attention task. Therefore, we calculated a 26263 repeated measures ANOVA on the response times with the factors congruency (congruent, incongruent), attention (global, local) and time (first, second or third run with the same instruction). The ANOVA revealed a significant main effect for congruency (F(1,7) = 36.198 p = 0.001) only. All other Taken together we found the typical Stroop effect with lower performance and longer response times for the incongruent compared to the congruent stimuli. However, it is very important to note that we corrected the individual reaction times of each run with a condition and run specific mean, such that the mean for congruent and incongruent responses was zero. As a consequence only fluctuations around a condition specific average reaction time were used to model the NIRS response (Analysis I) and predicted by the NIRS response (Analysis II and III).

Analysis I: Finite Impulse Response
The group average parameter estimates of the ER-HR revealed a classic hemodynamic response profile with a significant increase around 7 seconds after stimulus onset in most channels of oxygenated hemoglobin (HbO, Figure 5, red lines). The second set of regressors ( Figure 5, green lines), for which mean corrected reaction times of the single trial were modulated by the height of the regressors' amplitude, also showed significant effects.
Analysis II and III: Multivariate SVR Support vector regression revealed significant decoding of the reaction times between 4 and 8 seconds after stimulus onset which is around the observed time of the peak of the BOLD response as shown in Figure 6. There were no significant effects before stimulus onset which could have predicted single trial reaction times. The average inter-individual weighting vector reveals that mainly the channels from parietal areas contributed to regression (Figure 7). Please note that the neurophysiological interpretation of weight vectors of linear multivariate models can lead to wrong conclusions [50]. For example, channels that contain correlated noise can receive strong weights for removing the noise and increase the predictive power of the model. However these informative noise channels are only indirectly informative as they are used to 'clean' other channels. Further analysis of the spectral components (Analysis III) lead to significant effects in frequencies between 0.1 and 0.15 Hz (see Figure 8). The effect seen around this frequency range reflects the inter-trial-interval of the repetitive experimental design.

Discussion
In the present study, we used a Stroop attention paradigm to measure fluctuations in reaction times and their neural correlates using NIRS. We were able to decode reaction time variability from NIRS signals using several approaches. Using univariate analysis techniques, we found that variations in reaction times were correlated with HbO changes in frontal and parietal regions. Results of the FIR analysis suggest that the effects are higher in HbO as compared to HbR. The average event-related hemodynamic responses (ER-HR) follow the expected canonical hemodynamic response with significant peaks between 5-8 seconds in frontal and parietal areas. The event-related parametric modulation (ER-PAR) showed that the time of peak is related to the reaction time of the subject: For slower reaction times there are also later peaks of the hemodynamic response. Neural correlates of attentional slips have been reported previously using fMRI [5]. Our results show that even at the much lower spatial resolution and with hemodynamic responses only from the surface of the brain it is possible to decode attention changes.
We showed that decoding of single trial reaction times is feasible using spatio-temporal patterns of HbR/HbO changes. Significant group statistic results were gained in the time range of 6-9 seconds after stimulus onset ( Figure 6). This is also the time where the hemodynamic response is strongest and where we found strong univariate parametric modulations of the reaction times. However, we were not able to predict reaction times before stimulus onset. It has been reported previously that neural responses before stimulus onset effect responses to a stimulus [51,52]. However, usually the pre-stimulus effect is small and only found by averaging many trials.
It might be possible that the NIRS signal reflects a recognition signal of slow (and fast) response performance. In other words it might be possible that the subject's awareness of slow/fast responses was decoded. However, we think that this is very unlikely because we decoded the single trial reaction times (condition specific mean corrected) of a whole block. It seems extremely unlikely that participants are aware of the specific deviation of their responses from the condition specific mean for each of their given responses. Furthermore, the participants might be aware when they have responded very slow but for a successful decoding of the response times this 'awareness signal' would have to be correlated with the actual given response times. However, a temporal separation of the response and the signal that reflects the awareness of a slow/fast response would not be possible with fNIRS and therefore we can only speculate whether the observed fNIRS signal reflects a recognition signal of slow response performance. In further studies the combination of fNIRS with EEG with much higher temporal resolution could be used to approach specifically this interesting question.
Our decoding accuracy was significant, but not very high. One of the reasons for this could be that we measured vigilance fluctuations as the variation of reaction times during a task that was performed for roughly one hour. In contrast, other studies have used tasks where subjects have to monitor very rare events, and then used the hit rate as a measure of vigilance [40]. For us such a task was not practical because we aimed for a continuous and parametric model of vigilance. However, our choice might have a disadvantage because reaction times are influenced also by other, non attention-related factors such as motivation or motor fatigue. Thus, the significant but rather low decoding performance should be considered a lower bound of the potential to decode vigilance, and future studies might want to seek neural correlates of vigilance that are robust across multiple tasks. A second reason for the low decoding performance could be that NIRS might not have sufficient information for the high accuracies required in realworld applications. In future, simultaneous other measurements for example of blood pressure, heart rate, and eye movements could be included in the decoding and potentially enhance the decoding performance.

Summary
Taken together in the present study we show that neural correlates of vigilance can be assessed from the surface of prefrontal and parietal brain regions using NIRS. However, decoding performance on single trials is not good enough to allow for a continuous vigilance measuring device that could be of practical use yet. Therefore, further research with different tasks and simultaneous acquisition of NIRS with other physiological parameters could be fruitful.