Fig 1.
Cross-modal oddball task diagram.
The top row depicts the trial timing. The gray box attached to the cue illustrates the different cue properties by trial type. Each cue consisted of a visual and an auditory component. Standard visual cues consisted of a green circle, whereas unexpected visual cues were one of seven non-circular shapes shown in one of fourteen non-green colors. Standard auditory cues consisted of a 600Hz sine wave, whereas unexpected auditory cues were one of 120 individual unique birdsong segments. On a trial that contained an unexpected cue in one domain, the other dimension of the cue always contained the standard component (i.e., a cue was either unexpected in the visual or the auditory domain, never in both).
Fig 2.
Single-subject example of surprise-term construction for each model.
Top: Model 1 uses separate surprise terms for each sensory domain. In effect, the presence of a surprising event in one sensory domain does not inform the prior in the other sensory domain. Bottom: Model 2 uses a combined surprise term across both domains. In effect, all unexpected events, regardless of domain, influence the construction of the prior.
Fig 3.
Reaction time data from the cross-modal oddball task.
Significant individual comparisons (Bonferroni-corrected) are highlighted in red. For both unexpected auditory and unexpected visual cues, reaction times were slower compared to standard cues in Block 1. This effect wore off over time. White error bars denote the standard error of the mean, gray dots indicate individual subject condition means.
Fig 4.
Results from the whole-brain single-trial model fitting analysis described in Fig 2.
Each topography depicts the averaged standardized beta coefficient at each channel in the respective time window (x-axis) and model (plots A and B), as well as the M1-M2 model comparison (plot C). White areas denote channels at which the fit within the depicted time-window was non-significant (p < .00044). In A and B, red areas denote significant positive correlations between the respective model and the EEG data, blue areas denote significant negative correlations. In C, red areas denote higher correlations between Model 1 and the data compared to Model 2.
Fig 5.
Average channel event-related response to the three different cue types, plotted at the channels in which the winning model (separate surprise terms; Model 1) provided significantly better fit than the losing model (common surprise term).
Beige highlighting denotes the time window in which the winning model significantly fit the single-trial EEG response. This trial average illustrates that the time window in which the fit was significant contains the fronto-central P3 ERP to both unexpected auditory and visual cues. Shaded area around the ERP curves denotes the standard error of the mean.
Fig 6.
Split model fits of the within-domain surprise values, individually for each sensory domain.
Scaling and significance threshold is the same as in Fig 4 (p < .00044). This plot shows that both the auditory and visual unexpected cues contribute to the significant single-trial fit of the separate surprise-terms model in Fig 4A.
Fig 7.
Cross-task correlations between fronto-central activity during surprise processing and action-stopping.
A) The amplitude of the stop-signal P3 in the SST was positively correlated with the surprise-related P3 in the CMO task; this was the case for both for auditory (left) and visual (right) unexpected cues. B) Control analyses show that ERP amplitudes of similar processes are indeed positively correlated across tasks (illustrated by the posterior visual N1 to the imperative arrow stimuli in both tasks–i.e., the Go-signal in the SST and the target in the CMO task). Ruling out alternative explanations, there was no reliable correlation for different waveforms from different tasks (middle left; Target visual N1 to stop-signal P3), different waveforms from the same task (right; go-signal visual N1 to stop-signal P3), or the same waveform from different tasks (middle right; standard cue P3 to stop-signal P3). These control analyses demonstrate that there is no general relationship between ERP amplitudes within or across the same task or within the same region of cortex, unless related processes are active.
Fig 8.
Properties of the independent component selected to reflect the stop-signal P3 in the SST from the merged dataset analysis.
The left plot shows that the morphology of the stop-signal P3 based on all non-artifact components (i.e., the standard channel ERP) can be entirely reproduced using just one IC. This shows that the selection algorithm identified the appropriate component, accounting for the activity of the stop-signal P3. Furthermore, this IC shows the classic features demonstrated for the stop-signal P3 in the SST. Namely, the onset of the P3 was earlier on successful vs. failed stop trials (inlay on left plot), and was positively correlated with SSRT across subjects (right plot). Shaded area around the ERP curves denotes the standard error of the mean.
Fig 9.
A) A reanalysis of the single-trial model fitting analysis for the winning model (separate surprise terms cf. Fig 4A) using just the one IC that was selected to reflect the stop-signal P3 in the merged dataset. The significant association between fronto-central EEG activity following unexpected cues in the CMO task and the surprise model is retained when the data is reconstructed solely using that one ICA (out of ~17.1 overall ICs that were extracted per subject on average). B) For comparison, no significant association was found when the data were reconstructed based on the ~16.1 ICs that did not reflect the stop-signal P3.
Fig 10.
Stop-signal P3 split by phase of the SST experiment.
If the process underlying the stop-signal P3 was stop-signal-induced surprise, its amplitude should decrease in the second half of the experiment. Instead, the stop-signal P3 is nearly identical across the two halves of the experiment. Shaded area around the ERP curves denotes the standard error of the mean.