Electrophysiological markers of mind wandering

Ample evidence from scalp EEG studies has established a distinct set of electrophysiological signatures of mind wandering, which is promising for real-time detection [28–34]. Given that scalp EEG is comparably low cost, and can be implemented outside of the laboratory, these EEG measures could be ideal for real-time detection of this phenomenon in the real world. In particular, the P1 and N1 ERP components in response to visual and auditory probes in a target detection task are reduced during mind wandering [33, 35, 36]. This indicates that sensory-evoked responses to both visual and auditory inputs were significantly attenuated, suggesting that mind wandering disrupts external perception regardless of sensory modality. Similarly, several studies have documented a reduction of the P3 ERP component during periods of mind wandering relative to on task [12, 34, 37, 38], demonstrating an overall attenuation of higher-level cognitive processes. Together, these findings suggest that mind wandering periods are associated with reduced external processing as observed in changes in ERP amplitude. These studies are consistent with the executive function model of mind wandering, which posits that in order to facilitate mind wandering, our executive resources are disengaged from the external task and instead directed internally to our thoughts [2, 7].

Prior EEG studies have also reported changes in low frequency power during mind wandering. Specifically, Braboszcz and Delorme reported increases in theta activity and decreases in alpha activity during mind wandering [28]. Using a similar experimental design, van Son and colleagues [39] found a higher ratio of theta and beta activity in the frontal cortex during mind wandering. In contrast to these findings, others have reported increased frontocentral theta power during external cognition [40–42], whereas increased posterior alpha power has been implicated in mind wandering [23, 38]. These variable patterns in low frequency power as a marker of mind wandering may be driven by differences in stimulus modality, experimental manipulation of top-down processes, or the electrode sites at which power was measured. Importantly, given the overlapping information between low frequency power and ERP components (typically measured between 1-30Hz), and that ERP patterns are relatively more consistent across studies, the current study used ERP measures as features in the machine learning models.

Using EEG for mind wandering detection

Converging evidence points towards several reliable EEG correlates of mind wandering, and several studies to date have attempted to build detectors of mind wandering based solely on EEG measures. Kawashima and Kumano [43] used EEG signals (power and coherence in delta, theta, alpha, beta, and gamma frequency bands) to predict mind wandering intensity during a sustained attention to response task. They found that non-linear models using multiple electrodes resulted in higher prediction accuracy of mind wandering intensity than linear models using a single electrode. Jin and colleagues [44] extended this work by predicting mind wandering with a nonlinear model that generalized across tasks within individual subjects. Specifically, they used a support vector machine to predict mind wandering with EEG markers (including the P1, N1, and P3 ERP components, as well as theta and alpha power and coherence), reporting an average on task/off task classification accuracy of 60% that generalized across two visual tasks. This task generalization is noteworthy as it suggests that models trained on EEG markers may detect mind-wandering without needing to first train on new tasks. This group [45] also demonstrated that mind wandering is not dependent on and is quantitatively different from subject vigilance and current task demands, by reporting that a classifier trained through thought sampling outperformed classifiers trained through either vigilance or task demands. Dhindsa and colleagues [46] extended these findings by detecting mind wandering in real world settings. They recorded EEG activity during live lectures and used frequency band power measurements (theta, alpha, and beta) to achieve an average detection accuracy of 80–83%. Finally, Tasika and colleagues [47] employed a multi-step framework that leverages a J48 decision tree classifier and a support vector machine with a radial basis function kernel to detect mind wandering using EEG, and reported an average accuracy as high as 84.49% based on two individuals.

An important property that is not clearly addressed in previous work, however, is the generalizability of models across participants: a model that identifies an optimal set of features at the group level that can accurately predict attention states of another individual not in that group (i.e. person-independent). Although this approach typically results in overall lower classification performance, it allows for more flexible generalizability for real-time prediction. Previous attempts using machine learning and EEG measures exclusively to detect mind wandering used data from the same individuals to train and test the models. In other words, there was dependence within individuals, which means the models likely do not generalize well to new subjects. We aim to improve on mind wandering detection using only EEG by building models that generalize across individuals in a person-independent manner.

This proof-of-concept study examined whether machine learning models using EEG measures can detect mind wandering 1) within individuals and 2) across individuals. Using thought sampling during a target detection task, we asked subjects to report their attention state throughout the task as we recorded their EEG. Our study included EEG markers, specifically ERP components that have previously demonstrated reliable attentional differences, as features in two machine learning models. To our knowledge, this is the first study to establish that machine learning using EEG features is capable of detecting mind wandering both within individuals (i.e. person-dependent) and across individuals (i.e. person-independent).

To address this issue, we asked participants to perform an auditory target detection task while their EEG is being recorded. In order to obtain a measure of participants’ attention state in the moment, participants were asked to report their attentional state as on task or mind wandering at pseudorandom intervals throughout the task. We extracted EEG measures of interest (namely N1 and P3 ERP) as a function of the reported attention states. Using these EEG measures, we built linear and non-linear machine learning models to determine whether these measures can be used to classify attention states both within and across individuals, and if so, which models led to superior performance. For classification within subjects, we used each subject’s own EEG markers of attention state for prediction, which allowed us to maximize prediction accuracy for that individual. For classification across subjects, our models trained on one set of data and attempted to identify an optimal algorithm that can predict attentional state of individuals not part of the training group data. Together, these two approaches help determine whether machine learning with EEG measures can accurately predict mind wandering within and across individuals.

Subjects

Fourteen subjects participated in the experiment (9 females, 5 males; age range: 24–75, M ± SD = 51.9± 14.7). Although the sample size appears small, it is sufficient for addressing our primary aim of mind wandering detection. Specifically, it is comparable with previous studies using within-subject training and testing sets that included fewer than 20 subjects to detect mind wandering using EEG features (i.e. features that were derived from EEG data) [44, 46]. Further, if we can predict mind wandering with this sample size for the across-subject analyses, this finding provides a lower threshold necessary for accurately detecting mind wandering using EEG features. All subjects provided informed written consent and were reimbursed for their participation. This study was approved by the Institutional Review Board at the University of California, Berkeley.

Task stimuli and paradigm

Subjects sat in a dark room and performed an auditory target detection task [36]. They were presented with a series of standard tones (800Hz) and target tones (1000Hz) in a random order with probabilities of 0.8 and 0.2, respectively. There was a total of 1500 tones, 1200 of which were standard tones and 300 of which were target tones. Each sound was a pure tone presented at 75 dB SL that lasted 200ms, and the inter-trial interval was randomly jittered between 800-1200ms. Subjects were instructed to press a button to target tones as quickly and accurately as possible. They were asked to keep their eyes fixated on the cross in the center of the screen at all times. Reaction time to the target tones was recorded, and accuracy was considered across both tones (i.e. detection of target tones and correct rejection of standard tones). Attentional differences in behavioral measures were assessed using dependent samples t-tests.

During the task, we occasionally presented thought probes that asked subjects to report their attention state as either “on task” or “mind wandering.” To ensure subjects understood the meaning of these attention states, we provided them with clear and detailed definitions and examples. “On task” was defined as one’s attention being firmly directed towards the target detection task, and “mind wandering” was defined as one’s attention being oriented away from the task. For each thought probe, we extracted the 10 trials (approximately 15 seconds) prior to the probe, and labeled these 10 trials according to the subject’s response to the thought probe. For instance, if the response to one thought probe was “mind wandering”, then the 10 trials preceding that thought probe received a label of mind wandering, and are referred to as mind wandering trials thereafter. This time window has been previously used in ERP studies [12, 33, 37, 48] in order to maximize the number of trials that can be included to create a reliable ERP average while still maintaining a reasonable validity of the attentional report. Earlier studies have also provided detailed justification for using this time window [33, 34]. Each block ended with a thought probe. There was a total of 25 blocks, with varying block duration (i.e. 45 to 75 seconds) to prevent subjects from anticipating thought probe occurrence. Therefore, a maximum number of 250 trials (10 trials x 25 blocks) were included in subsequent analyses for each subject.

Despite the shortcomings of the thought sampling approach, it serves to provide the labelled attention reports for our supervised machine learning models (as described below). Therefore, thought sampling remains a valuable tool for validating machine learning models in mind wandering. Once sufficient evidence accumulates that validate machine learning models across different experimental paradigms, we can strive to reduce reliance on this measure in future studies.

EEG acquisition and preprocessing

EEG was recorded reference-free continuously from 64 active electrodes mounted on a cap using the Biosemi ActiveTwo system (www.Biosemi.com). Electrodes were placed according to the International 10–20 system. Continuous EEG data were amplified and digitized at 1024 Hz, and bandpass filtered between 1Hz and 50Hz. Vertical and horizontal eye movements were recorded from electrodes above and below the right eye and two electrodes placed at the right and left outer canthus. EEG data were down-sampled to 512 Hz, then high-pass filtered at 1 Hz (as this is ideal for independent component analysis [49]), and notch filtered at 60 Hz. Electrodes with excessively noisy signals were removed and replaced with an interpolation from neighboring electrodes using spherical spline interpolation [50]. Independent component analysis was used to detect and remove eye movement and muscle artifacts. Continuous EEG data were then segmented into 3000ms epochs, beginning at 1000ms prior to stimulus onset. Each trial was visually inspected for any remaining artifacts, which were further manually removed. Common average reference was then applied to each subject’s data prior to ERP analysis. EEG data pre-processing and analysis were adapted from Kam and colleagues [51] and performed using EEGLAB [52, 53] and custom Matlab scripts.

ERP analysis

EEG signals were bandpass filtered at 1-15Hz for ERP analysis [54–56]. All ERPs were quantified by the peak amplitude measure relative to a -200 to 0 pre-stimulus baseline. For N1, we extracted the minimum amplitude in the 80–120 ms post-stimulus time window over fronto-central midline sites (FC1, FCz, FC2), where N1 is typically maximal and also maximal in our data. For P3, we extracted the maximum amplitude in the 400–600 ms post-stimulus time window over parietal sites (P1, Pz, P2), where P3 is typically maximal and also maximal in our data.

For each ERP measure, we averaged across the channels and time points of interest as described above. We considered up to the 10 trials (excluding artifactual trials) preceding the thought probe at the end of each block, and categorized them according to the reported attention state. In order to ensure our data are consistent with previous findings, we first implemented univariate analyses to examine attentional effects in the N1 and P3 ERP components. In particular, we statistically compared the peak ERP amplitudes between on task and mind wandering states by implementing repeated measures ANOVAs for each ERP component (i.e. N1 and P3) to examine attentional effects (i.e. on task and mind wandering) while taking into account tone differences (i.e. standard and target tones). Paired samples t-tests were conducted post hoc to examine attentional effects separately for each tone. These analyses serve the purpose of ensuring that ERP patterns across on task and mind wandering states are consistent with previous studies.

For machine learning analyses, we extracted information at the block level. Specifically, we computed the mean and standard deviation across up to 10 trials for each block in order to obtain a stable ERP measure (via averaging across trials per block) and have sufficient data points per subject to input into the machine learning model (via deriving one input per block). The alternative options are suboptimal, including measuring ERPs at the single trial level which results in unreliable estimates of ERP, or averaging across all blocks to yield one grand average value per subject which results in insufficient inputs for machine learning models to classify attention states. Given the temporal fluctuations of our attention state, our approach allows us to account for the ebb and flow of attention throughout the task and its corresponding changes in the electrophysiological measures across time within individuals. Accordingly, for each block, we extracted the following features: 1) the mean across the 10 preceding trials for each ERP measure (the N1 minimum amplitude and the P3 maximum amplitude), as an index of the magnitude of electrophysiological response, and 2) the standard deviation across the 10 trials for each ERP measure, as an index of the variability in our response to external events. In essence, each subject will have up to as many inputs as blocks completed (i.e. 25) for each ERP component (i.e. N1 and P3) and descriptive measure (mean and standard deviation of ERP peak amplitudes across 10 trials per block) into the machine learning models.

Statistical analysis

To predict attention states, we included the aforementioned ERP measures as features into a machine learning model that makes a binary classification: on task (0) or mind wandering (1). In particular, we derived the mean and standard deviation of the N1 and P3 peak amplitudes of the last 10 trials per block. For these analyses, we focused on ERP amplitudes in response to standard tones only since there were very few target tones (i.e. on average two) occurring within the last 10 trials within a block. This low number of target tones not only yields an unreliable estimate of the ERP response to target tones, but it could reduce generalization across individuals. Using Scikit-learn (version 0.20.2), we built two machine learning models to discriminate between the two attention states, including one linear model (i.e. linear logistic regression) and one non-linear model (i.e. a support vector machine (SVM) with a radial basis function (RBF) kernel). We computed the mean amplitude across all blocks in the on task state and subtracted this mean from the data for both attention states; this additional step normalizes the data, which makes the two conditions more comparable across different subjects [57]. This normalization was performed independently for each feature and each subject. We adopted this approach so that our machine learning models would likely be less impacted by individual differences in the absolute values of each EEG measure.

Behavioral performance

Subjects reported mind wandering 55% of the time (range: 20%– 88%, SE = 5.3%) and being on task 45% of the time. This is consistent with the typical breakdown of self-reported attention states in the literature for this type of task [34, 35, 37]. Mean reaction time and accuracy are shown in Fig 1. There was a trend for slower response time during mind wandering (M = 535 ms, SE = 14 ms) compared to on task periods (M = 511 ms, SE = 14 ms; t(12) = -1.94, p = 0.076). However, accuracy did not differ between attention states for mind wandering (M = 0.9619, SE = 0.030) compared to on task (M = 0.9623, SE = 0.028; t(12) = 0.098, p = 0.924).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Behavioral results as a function of attention state.

(A) Mean response time was slower during mind wandering (p = 0.076). (B) No difference was observed in accuracy between the two attention states. Error bars = standard error of the mean; OT = on task; MW = mind wandering.

https://doi.org/10.1371/journal.pone.0251490.g001

Univariate analyses on ERP components

In order to check the veracity of the attentional reports, we first evaluated whether our ERP measures reveal attentional differences that are consistent with previous studies. Specifically, if our ERP measures show reduced amplitudes during mind wandering, this would provide corroborating evidence for the attentional reports. To assess conditional differences in ERP measures, we implemented repeated measures ANOVA with attention (on task and mind wandering) and tone (standard and target) as within subject factors, separately for the N1 and P3 ERP components. These results are reported in Table 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. ANOVAs on ERP components.

https://doi.org/10.1371/journal.pone.0251490.t001

As expected, the main effect of tone was significant for both the N1 and P3, with increased amplitude during target tones relative to standard tones. For N1, there was a near significant main attentional effect, characterized by reduced N1 during mind wandering relative to on task periods. Although the attention x tone interaction was not significant, we implemented follow up analyses using paired samples t-tests to examine whether attentional effects varied for each tone as was observed in Fig 2. Consistent with previous findings of attentional effects in sensory ERP components [33, 36], our analyses revealed a significant attentional effect for the N1 in response to target tones (t(13) = -2.83, p = 0.014) but not standard tones (t(13) = -0.72, p = 0.485). Neither the main attention effect nor interaction effect were significant for the P3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Grand average ERP waveforms.

N1 was averaged across FC1, FCz and FC2 (left panel), whereas P3 was averaged across P1, Pz and P2 (right panel). Univariate analyses indicate reduced N1 in response to target tones during MW relative to OT periods. OT = on task, MW = mind wandering.

https://doi.org/10.1371/journal.pone.0251490.g002

Machine learning

Person-dependent classification performance.

Individual accuracy for subjects with the non-linear SVM with RBF kernel ranged from 0.383 to 0.960 (M = 0.643, SE = 0.044), whereas accuracy for the linear logistic regression model ranged from 0.375 to 0.843 (M = 0.621, SE = 0.038). The range of accuracy values observed in our study, as well as the variability in accuracy rates across individuals, are similar to accuracy values reported in prior work using EEG measures for predicting mind wandering [44, 46]. Although accuracy might be useful for comparing findings across different studies, this metric can be biased with class imbalance (as previously mentioned); therefore, it is more informative to examine the AUC and MCC values for individual subjects.

Both AUC and MCC metrics also demonstrate, on average, above-chance performance for the SVM and logistic regression models. AUC ranged from 0.452 to 0.995 (M = 0.715, SE = 0.0457) for the SVM and 0.291 to 0.858 (M = 0.635, SE = 0.0491) for the logistic regression model. MCC ranged from -0.162 to 0.919 (M = 0.289, SE = 0.0792) for the SVM and -0.287 to 0.694 (M = 0.210, SE = 0.0758) for the logistic regression model. Machine learning performance as indexed by AUC and MCC for each individual subject is shown in Fig 3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Model performance per subject, as measured by AUC and MCC.

AUC performance for each subject for both models (SVM and logistic regression) is shown in top panel; chance is noted by the black horizontal line at 0.5. MCC performance for each subject is shown in bottom panel; chance is noted by the black horizontal line at 0. AUC = area under the curve; MCC = Matthews correlation coefficient; SVM = support vector machine.

https://doi.org/10.1371/journal.pone.0251490.g003

Person-independent classification performance.

In this method of classification, we aimed to generalize the model across subjects by using leave-one-subject-out cross-validation. Both the nonlinear SVM and logistic regression models performed above chance (MCC > 0, AUC > 0.5). Accuracy ranged from 0.514 to 0.750 (M = 0.591, SD = 0.070) for the SVM, and from 0.461 to 0.752 (M = 0.588, SD = 0.093) for the logistic regression model. MCC values ranged from –0.026 to 0.446 (M = 0.206, SD = 0.152) for the SVM and –0.056 to 0.449 (M = 0.196, SD = 0.169) for the logistic regression. Finally, AUC values ranged from 0.485 to 0.731 (M = 0.613, SD = 0.085) for the SVM and 0.468 to 0.739 (M = 0.609, SD = 0.096) for the logistic regression model. Overall, the nonlinear SVM showed the best performance, as indicated by higher scores in accuracy, AUC, and MCC, as compared to logistic regression. Although there is clear variability in the evaluation metrics across participants (as is common for other real-time detectors [46]), we suggest the average model performance faired quite well for most individuals. A summary of the results for each model are reported in Table 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Model evaluation of person-independent classification performance.

https://doi.org/10.1371/journal.pone.0251490.t002

The ultimate goal of a mind wandering detector is to assess the phenomenon in real time for measurement and interventions; thus precision (i.e. accurately classifying instances of mind wandering as mind wandering) is a priority of the model. We therefore evaluated the confusion matrices of the best models to determine the type of errors the models made (as shown in Table 3). Neither model exhibited any skew towards sensitivity (i.e. true positives for predicting mind wandering) or specificity (i.e. true negatives or correct rejections). Taken together, the SVM model outperformed the logistic regression model. Not only did the SVM model show higher sensitivity and specificity, it also showed higher values for AUC and MCC, which are metrics that are robust against the presence of imbalanced classes.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Confusion matrices of person-independent models.

https://doi.org/10.1371/journal.pone.0251490.t003

Feature analysis.

To further examine each feature individually, we created models that were trained on only one feature at a time (Table 4). Given that SVM outperformed logistic regression, we implemented these analyses with the SVM model only. This post-hoc analysis allowed us to identify the feature that is most effective in predicting attention states. Our results indicate that most features performed slightly above chance, with the peak P3 amplitude showing the best performance. Notably, none of the features individually performed as well as the complete model.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Model performance for individual features.

https://doi.org/10.1371/journal.pone.0251490.t004