https://orcid.org/0000-0003-1247-2577
Update
27 Feb 2019: Smout CA, Tang MF, Garrido MI, Mattingley JB (2019) Attention promotes the neural encoding of prediction errors. PLOS Biology 17(2): e2006812. https://doi.org/10.1371/journal.pbio.2006812 View update
Figures
Abstract
Prediction error is a basic component of predictive-coding theory of brain processing. According to the theory, each stage of brain processing of sensory information generates a model of the current sensory input; subsequent input is compared against the model and only if there is a mismatch, a prediction error, is further processing performed. Recently, Smout and colleagues found that a signature of prediction error, the visual (v) mismatch negativity (MMN), for a fundamental property of visual input—its orientation—was absent without endogenous attention on the stimuli. This is remarkable because the weight of evidence for MMNs from audition and vision is that they occur without endogenous attention. To resolve this discrepancy, we conducted an experiment addressing 2 alternative explanations for Smout and colleagues’ finding: that it was from a lack of reproducibility or that participants’ visual systems did not encode the stimuli when attention was on something else. We conducted a similar experiment to that of Smout and colleagues. We showed 21 participants sequences of identically oriented Gabor patches, standards, and, unpredictably, otherwise identical, Gabor patches differing in orientation by ±15°, ±30°, and ±60°, deviants. To test whether participants encoded the orientation of the standards, we varied the number of standards preceding a deviant, allowing us to search for a decrease in activity with the number of repetitions of standards—repetition suppression. We diverted participants’ attention from the oriented stimuli with a central, letter-detection task. We reproduced Smout and colleagues’ finding of no vMMN without endogenous attention, strengthening their finding. We found that our participants showed repetition suppression: They did encode the stimuli preattentively. We also found early processing of deviants. We discuss various explanations why the earlier processing did not extend into the vMMN time window, including low precision of prediction.
Citation: Male AG, O’Shea RP (2023) Attention is required for canonical brain signature of prediction error despite early encoding of the stimuli. PLoS Biol 21(6): e3001866. https://doi.org/10.1371/journal.pbio.3001866
Academic Editor: Ben Seymour, University of Cambridge, UNITED KINGDOM
Received: September 24, 2022; Accepted: May 11, 2023; Published: June 20, 2023
Copyright: © 2023 Male, O’Shea. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Abbreviations: EDRP, early deviant-related positivity; EEG, electroencephalogram; ERP, event-related potential; FIR, finite impulse response; ICA, independent component analysis; MMN, mismatch negativity; PCA, principal component analysis; PO, parieto-occipital; ROI, region of interest; vMMN, visual mismatch negativity
Our senses are flooded by information. For example, the 4 million cones of the human retina encode a staggering 101,200,000 bits of information at any instant if their responses to light were binary (which they are not) [1]. Still, we do not experience a flood—only a stream—of information to which we pay attention or of which we become conscious [2]. What we see does not require such an impossible burden of encoding because it is highly redundant, allowing prediction of the state of one photoreceptor at any instant from its state the instant before and from the state of its neighbors. This seminal idea developed into predictive-coding theory, first of the retina [3] and then of the brain’s hierarchical sensory systems [4] and ultimately to the whole brain [5–11].
Predictive-coding theory is a leading theory of how the brain deals with sensory input [12]. Friston [7,13–15] described the theory based on energy conservation and free-energy principles: It is that the brain uses past, bottom-up, sensory information, along with top-down information (such as prior probabilities, expectations, and attention) to generate predictive models of sensory input at various levels of the sensory pathway, linked by feedforward and feedback connections. To ensure good predictions, precision of predictions from previous bottom-up and top-down influences and from the success of previous predictions affect whether, or the degree to which, the model is updated [16–19]. For example, a prediction error for a small irregularity after high-precision predictions should prompt updating of the model, whereas the same irregularity after low-precision predictions may not [16].
We studied the visual mismatch negativity (vMMN) as our signature of prediction error [20]. The vMMN is an event-related potential (ERP) component from electroencephalography. Stefanics and colleagues [20] have reviewed its hundreds of studies: It occurs when a rare, unpredicted, deviant, visual stimulus occurs after a sequence of identical, standard, visual stimuli in a so-called oddball sequence. It is greater ERP negativity from parieto-occipital (PO) recording sites for deviants than for standards, occurring between 150 ms and 300 ms after the onset of the deviant. It is supposed to occur without endogenous attention on the deviant property of the stimuli.
There are 2 measures of the vMMN [20]. One is the classic vMMN, calculated as the difference between the ERP to deviants minus that to standards. The classic vMMN involves comparison of physically different stimuli and includes the greater neural adaptation of the frequently repeated standards than of the occasionally repeated deviants. The other is the genuine vMMN, calculated as the difference between the ERP to deviants minus that to some control (usually the equiprobable control [21,22]) stimuli, physically identical to the deviants and with equal neural adaptation, hence excluding it [23]. It has been used for hundreds of human ERP and animal single-cell experiments for various modalities. We focus on the genuine vMMN.
Recently, Smout and colleagues [24] found no ERP evidence of the genuine vMMN without endogenous attention to unexpected changes in orientation. In their attended condition, they showed each participant displays consisting of an annular Gabor patch (gratings of blurry dark and light bars) with a central gray patch containing a concentric black spot. They asked each participant to look only at the spot but to pay attention to the bars and to press a button on rare occasions when their spatial frequency increased. In their unattended condition, the same participant viewed the same displays but paid attention to the spot and pressed a button on rare occasions when its contrast decreased. The difficulty of the 2 tasks was equated for each participant in prior psychophysical testing.
Smout and colleagues’ stimuli of interest were the Gabor annuli. Participants viewed different-length sequences of displays of identical annuli—standards—followed by an otherwise identical annulus but with bars differing in orientation by ±20°, ±40°, ±60°, or ±80°—a deviant. On the next display, the deviant annulus was repeated, commencing a new sequence for which it now served as the standard—the so-called roving-standard paradigm [25]. To measure the genuine vMMN, Smout and colleagues used the equiprobable control.
Smout and colleagues found a genuine vMMN from PO electrodes to unexpected changes in the orientation of the bars in the attended condition (between about 170 ms to about 300 ms), but not in the unattended condition (their S1B Fig). They did find genuine, deviant-related responses in the unattended condition: a negativity from about 330 to 430 ms from a midline central electrode (Cz) and from about 300 to 470 ms from a midline frontal electrode (Fz). But these are too late to be considered vMMNs, or any sort of prediction error from such an early brain-processing feature of visual input as orientation [26].
Smout and colleagues confined their remaining analyses to their attended condition to answer questions about the timing and orientation tuning of prediction error. But the absence of a genuine vMMN without endogenous attention was surprising. Smout and colleagues did not discuss it except to cite one study for a similar finding in the auditory modality [27]. To resolve this discrepancy, we conducted an experiment addressing 2 alternative explanations for Smout and colleagues’ finding: that it was from a lack of reproducibility or that participants’ visual systems did not encode the stimuli when their endogenous attention was on something else.
We made some changes to improve our study’s ability to explore these alternatives. Smout and colleagues’ stimuli were Gabor annuli with an outer diameter of 11° diameter and an inner diameter of 0.83°. Following Smout and colleagues’ argument that bigger is better, we doubled the size of the outer diameter of our Gabor patches. Smout and colleagues’ inner gray patch interrupted the central bars over a region of about the size of the human foveola to which about 20% of the processing of the visual system is devoted [28]. To include as much as possible of that area, we eliminated any central gray patch, making our central bars continuous except for a very small area occupied by the lines of small task-relevant fixation letters.
We also omitted an attended condition, eliminating any possible confound in Smout and colleagues’ design from participants’ moving their eyes to the bars [29], thus bringing them onto central vision. We also analyzed clusters of electrodes around the single electrodes Smout and colleagues reported. This was to avoid issues related to variability in placement of single electrodes [30].
To look for evidence that participants encoded the orientation of the Gabor patches, we searched for repetition suppression: attenuated brain activity to repeated standard stimuli [31,32]. For Gabor patches similar to those used by Smout and colleagues and by us, this attenuation appears in various components of ERPs, as late as the P2 [32] and as early as the P1 at PO electrodes [33].
We found no genuine vMMN to unattended orientation deviants even though participants encoded the stimuli.
Results
Participants looked at a central, desynchronized stream of letters and pressed a key whenever an X appeared (Fig 1A and 1B). Desynchronized from the letter stream, Gabor patches appeared with, in roving-standard blocks, occasional changes in orientation after varying numbers of repetitions of the same orientation and, in equiprobable-control blocks, randomly selected orientations on each trial (Fig 1C and 1D).
(A) Screenshots from the experiment of a Gabor patch tilted 8° clockwise from vertical (0°), with a superimposed fixation letter. (B) Magnified views of a sequence of target letters superimposed on the Gabor patch (first 2 images) or on the gray screen during the interstimulus interval. The participants’ task was to press a key whenever an X appeared. Each letter was shown for 600 ms. (C) Illustration of roving standard sequences in which the first few trials of one sequence are 8°… 8°… 8°… 68° (i.e., +60° orientation deviant for the first sequence), then…68°… 68°… 68°… 68°… 53°… (i.e., −15° orientation deviant for the second sequence). Standards are outlined in green; deviants are outlined in blue. The stimulus preceding a deviant is framed with 2 black chevrons. (D) Illustration of matching equiprobable controls in which we replaced standard stimuli—except for the ones immediately preceding a deviant from the corresponding roving standard sequence—with Gabor patches whose orientation was randomly chosen among the 12 possible orientations. All orientations appeared equally (8.3%) often. The stimulus in the sequence (also framed with 2 black chevrons) preceding the one that would have been the deviant in the corresponding roving standard sequence had an identical orientation to that from the corresponding roving standard sequence. The control (outlined in orange) also had the same orientation as that of the deviant from the corresponding roving standard sequence. Gabor patch onset is indicated on the timeline. Stimuli appeared for 80 ms. The interstimulus interval was 280 ms.
Did our participants attend to the letter-detection task, thereby ignoring the Gabor patches?
We checked whether our participants attended to the central, letter-detection task, thereby ignoring the Gabor patches. The mean hit rate for detecting the target X in the central letter stream was 98.22% and the mean false alarm rate was <0.1%, showing that participants paid attention to the task and did very well on it. There were no significant differences in performance of the task on roving-standard and equiprobable-control blocks (details of analyses in S 1 Supporting Information—Behavioral Results; S1 Data).
Did we find a vMMN to unpredicted orientation changes?
We did not find a genuine vMMN to unpredicted, unattended orientation changes. There is no hint of any genuine negativity in the vMMN time range to any unexpected orientation changes from any electrode cluster (we give details of our analyses, including Bayesian statistical tests, in S3 Table; S2 Data). We show the ERPs and the difference waves in Fig 2 (data in S3 Data), formatted similarly to those of Smout and colleagues. For simplicity, we give our results only for the ±60° orientation change, 10° larger than the average orientation change Smout and colleagues used. We give the ERPs and the difference waves for our other 2 orientation changes, ±15° and ±30°, in S1 and S2 Figs. Those difference waves also show no evidence of the vMMN. Preliminary inspection of ERPs confirm no vMMN for deviants in any length sequence of standards, i.e., we replicated Smout and colleagues’ finding no vMMN to unpredicted unattended changes in orientation, strengthening their finding (details in S1 Supporting Information—vMMN). To take another approach to describing our data, we used temporal principal component analysis (PCA; [34]). We found no component that was consistent with a vMMN.
(A) Grand average ERPs from various electrode clusters around the electrodes Smout and colleagues reported. (B) Difference waves for classic (60-degree deviant minus standard) and genuine (60-degree deviant minus 60-degree control) deviant-related activity. The arrowed components show the only genuine EDRP. Horizontal gray bars illustrate the time window in which Kimura and Takeda [39] found the largest deviant-minus-control difference (i.e., genuine vMMN) for 32.7° orientation deviants. Mean amplitudes from this time window were analyzed using Bayesian replication tests (results in S3 Table). The lighter colors surrounding the difference waves give ±1 standard error of the mean. We used the same y-axis scales as Smout and colleagues [24] (data in S3 Data). EDRP, early deviant-related positivity; ERP, event-related potential; vMMN, visual mismatch negativity.
We did find genuine, deviant-related activity outside the vMMN time range: a positivity we call the early deviant-related positivity (EDRP) from PO electrodes about 90 ms after onset of unpredicted orientation changes. PCA showed that this positivity was from the P1. Fig 3 illustrates these findings (details of analyses in S1 Supporting Information—Deviant-related positivity; data in S4 Data).
(A) PCA details of the P1. (B) PCA details of the N1. The topographic maps in leftmost column show the combined activity from deviant and control trials at peak latency of the P1 (top) and N1 (bottom). In the second column, dipoles of each component help to show that the pronounced negativity at the M PO reflects the combined negative pole of both P1 and N1 components converging in close temporal and spatial proximity. In the third column, we show the component loadings (scaled by SD) (thick black line) relative to all other components (thin multicolored lines). This illustrates the component’s contribution to the overall evoked activity recorded from the scalp. In columns 4 to 6, we show the scores for deviant (orange) and control (purple) trials for each magnitude of deviance (15°, 30°, 60°) at the left (L), midline (M), and right PO region. Error bars depict ±1 standard error. Statistical analyses show that the deviant-related positivity at PO regions in Fig 2 are statistically significant for deviant-minus-control positivity for the P1 only (details in the Supporting information: S2.1; data in S4 Data). PCA, principal component analysis; PO, parieto-occipital.
Smout and colleagues showed no sign of this early component in their unattended condition. This could be because they obscured the bars of the Gabor patch from central vision by their 0.83°-diameter gray patch, unlike in our study in which the bars crossed central vision.
Did we find repetition suppression?
We did find repetition suppression. To search for it, we used PCA to identify plausible components and then tested their component scores against sequential position of preceding standards from 2 to ≥9. We needed to combine sequences showing ≥9 standards to have enough data for a reasonable average for that value. Its average was 10.36; we used that value in our regression model; it is the abscissa of that point in the graphs of Figs 4 and S3 (data in S5 Data). We found 2 components showing repetition suppression—the N1 with a peak latency of 94 ms (S3 Fig) and more clearly in the P2 with a peak latency of 268 ms (Fig 4).
P2 PCA scores are shown separately for the left (L), midline (M), and right (R) PO ROIs. Regression equations show the dotted lines in the data for standard and random stimuli. Asterisks denote the linear regression significance: *p < .05, **p < .01, ***p < .001. Error bars show 1 SE. We show +1 SE for standard and −1 SE for random stimuli to avoid overlap (data in S5 Data). PCA, principal component analysis; PO, parieto-occipital; ROI, region of interest; SE, standard error.
Fig 4 shows that the PCA P2 scores became significantly less positive with increasing sequential position (i.e., repeated presentations) of standards (green points and regression lines). The P2 scores to random changes (black points and regression lines) became slightly more positive with sequential position, but not significantly. The results from the standard stimuli show clearly that our participants encoded them, so the absence of a vMMN cannot have been from lack of encoding.
Discussion
The current study has 3 important findings:
- When unexpected, unattended changes to only the orientation of visual input occur, the typical brain signature of prediction error, the genuine vMMN, does not occur. This confirms the similar finding by Smout and colleagues, suggesting that theirs is not due to any reproducibility error.
- Participants encoded the stimuli by showing repetition suppression. The encoding extends into the time range of the vMMN, as others have reported [32,33,35,36].
- There is a sign of prediction error about 90 ms after onset of the stimuli: the EDRP.
Below, we consider why unattended, unexpected changes of orientation do not seem to produce a well-accepted neural signature of prediction error, the vMMN, and why they seem to produce an EDRP. We also consider the implications of our findings for existing models of attentional control and stimulus selection.
Why no vMMN but an EDRP?
Absence of a genuine vMMN to unattended, unexpected changes in orientation is consistent with 5 experiments including ours [24,29,37,38], but inconsistent with 4 [22,39–41]. We consider 5 possible explanations:
- There really is no genuine vMMN to unexpected, unattended orientation changes, and the 4 out of 9 studies that reported it are examples of publication bias or methodological errors.
- There is a genuine vMMN to unexpected, unattended orientation changes, and the 5 studies that failed to find it suffered from some yet-to-be-determined methodological differences or problems.
- Because the orientation changes were not attended, they were processed early, but not for long enough to engage the vMMN mechanism.
- Endogenous attention is necessary for the vMMN.
- The vMMN is affected by the EDRP.
1. No genuine vMMN to unexpected orientation changes?
A 44% hit rate across studies for finding a genuine vMMN to unexpected, unattended orientation changes is something to take notice of. But it is possible there are studies reporting no vMMN that never made it into print, which would reduce that rate. The vMMN literature does indeed show signs of publication bias. For example, of the 22 orientation-vMMN published studies, including those only on the classic vMMN, we are aware of only 4 reporting no vMMN for a hit rate of 82%. All but one [29] also reported a different condition or experiment in which a vMMN was found, suggesting that experiments in which all conditions failed to show a vMMN are either not submitted for publication or submitted and rejected. Such a bias would make it difficult to identify the experimental parameters required to resolve whether the genuine vMMN to orientation changes occur without attention.
2. Methodological differences or problems?
We considered whether our experimental parameters were responsible for our failure to find a genuine vMMN. We can rule out that our:
- Orientation differences (15° was our smallest) were too small, because observers can discriminate orientation differences between gratings of about 0.4° [42]. Moreover, the experiments that did find a genuine vMMN tested an orientation difference of 30° to 36° [22,39,43–45], whereas we and Smout and colleagues fruitlessly tested some much larger orientation differences.
- Presentation time (80 ms) was too short, because others have shown genuine vMMNs for other unattended, more complex deviants with presentation times as short as 17 ms [46].
- Our interstimulus interval (280 ms) was too short because others have shown genuine vMMNs with intervals as short as 210 ms [47].
- Our use of the roving-standard paradigm was responsible because others [24,48] did find a genuine orientation vMMN using it. Moreover, that paradigm has been used successfully to show MMNs to various deviants [9,49–51].
In a review of the literature, we noted that researchers generally failed to isolate individual low-level features, such as orientation, from other, confounding factors including eye movements that are capable of changing simple orientation changes into more complex visual changes [29]. Even Smout and colleagues’ study suffered from this, with very careful control of fixation in their unattended condition (the participants’ task was to detect contrast changes in a small central spot, requiring them to look at it) but not in their attended condition (the participants’ task was to detect spatial-frequency changes in the bars of the oriented stimuli, 0.42° away from the fixation spot). Male and colleagues [29] have shown evidence that participants’ eyes do stray away from central fixation when the task-relevant information is elsewhere, despite instructions to keep fixation central.
A potential limitation of our study is that we cannot discount the possibility that participants found the letter task so easy that endogenous attention, or even overt attention, drifted away from fixation. If so, we would anticipate some evidence of a vMMN given that Smout and colleagues found a mismatch signal when participants attended to the stimulus of interest. We did not find any such evidence. Additionally, if attention or distraction were different for roving-standard versus equiprobable-control sequences, we would anticipate differences in the ERP waveforms for deviant and control stimuli, if not in task performance; however, we found both to be markedly similar. We take this as a good indicator that our control condition, like Smout and colleagues’, was suitably designed for exploring unattended irregularities in low-level properties of visual input.
3. Unattended orientation changes are processed early, but not in the vMMN range?
Potentially, unattended orientation changes are processed early but not for long enough or for far enough in the visual pathways to engage the vMMN mechanism. This could explain our finding of the EDRP, our findings on repetition suppression, and our (and Smout and colleagues’) failure to find a vMMN to unattended stimuli. It would also be consistent with the theory that although unattended sensory inputs are fully preserved in the early, feedforward pathways for about 200 ms, only attended inputs are maintained for further processing in later, large-scale, recurrent networks for consciousness [52,53]. However, our and other researchers’ experimental data show that the repetition suppression we observed overlaps with the time range of the vMMN as does the processing of unattended inputs.
There is indeed experimental evidence that processing of unattended visual stimuli diminishes with time [54–56]. For example, Moerel and colleagues [56] showed participants superimposed rectangular gratings of different orientations and colors (blue and orange) for 100 ms; participants pressed a key when a cued grating (e.g., 90°, blue) appeared in a particular sequence of trials including all possible combinations of orientations and colors. They analyzed EEG data only from trials well before or after trials containing a cued or partially cued grating (e.g., matched the orientation but not the color). They trained pattern classifiers to discriminate 2 presented orientations 90° apart. Moerel and colleagues found that classifier accuracy was above chance for uncued orientations from about 80 ms after onset but diminished with time, whereas accuracy for cued orientations diminished less with time.
But the times for which unattended stimuli yield significant activity are all more than 400 ms after onset [54–56]. This is well into, and past, the vMMN range, from 150 ms to 300 ms. Even our P2, repetition-suppression result, with peak activity at 268 ms, is well beyond the notional time at which feedforward inputs are theoretically meant to give way to recurrent processing. We conclude that the processing of our unattended stimuli, both standards and deviants, endured well into the vMMN range.
4. Endogenous attention is necessary for the vMMN?
One could argue that we and Smout and colleagues did not find a vMMN because endogenous attention is necessary for it. Auksztulewicz and Friston [27] have made this proposition for auditory input. Doing so seems to us to undermine one of the most frequently cited evolutionary purposes of prediction error: to monitor one’s immense sensory input preattentively and to select only those that elicit prediction errors. We have no dispute with the notion that endogenous attention serves to increase the precision of prediction errors, as Smout and colleagues and others have argued [16,17,27], thereby engaging the vMMN mechanism. The neural mechanism is via regulation of the postsynaptic gain of error units undertaking the top down-sensory comparison and production of prediction errors.
5. The vMMN is affected by the EDRP?
We found evidence of an early prediction error, the genuine EDRP, at the P1, about 90 ms after onset of the Gabor patch. Of course, we need to be cautious about the EDRP’s reliability. Although we have found a similar early genuine, deviant-related positivity for orientation and for other low-level stimulus features [29] and others have reported early positivities for deviants either with a vMMN [57–60] and similar [61] or without [38], Smout and colleagues showed no sign of an EDRP. Theories on the early positivity range from memory-based change-detection, to changes in stimulus eccentricity [61], to hemi-field mediated deviance detection [60].
Nevertheless, we can think of 3 reasons why presence or absence of the vMMN might be affected by the EDRP:
- Energy efficiency,
- Stimulus complexity, and
- Precision of prediction.
Energy efficiency.
One feature of predictive-coding theory is that it is designed for energy efficiency [7,14,15]. It is inefficient to respond to a prediction error at one level of the visual system if that error had been responded to at an earlier level. To decide whether this is reasonable, we had recourse to the better-studied auditory system.
When there are unexpected, unattended changes to auditory input, an MMN occurs from about 150 to 250 ms after onset [62–64]. More recently, ERP evidence has been found for processing of changes in physical properties of auditory input as early as 40 ms: the middle-latency response [65–70]. It might be akin to our EDRP.
However, one important characteristic of the middle-latency response is that it is always followed by the MMN, possibly because model updating is accomplished only by the MMN. This is consistent with predictive coding theory’s harmony with evolutionary reasons. A prediction error requires a listener to assess whether a preattentive stimulus represents a threat or a mating opportunity, thereby converting the stimulus into one demanding exogenous attention, yielding later ERP activity, in this case the MMN.
The visual EDRP does not show the same properties as the auditory middle-latency response: We found no hint of a vMMN after the EDRP. It will remain unknown whether the EDRP performs model updating until the EDRP is replicated and scrutinized for evidence of model updating. Although our study is not designed for revealing model updating, we hope that our findings will encourage studies better suited in pursuing the answer to this question.
Stimulus complexity.
If the EDRP for simple features of visual input involves both detection of a prediction error and updating of the predictive model, it is possible that more complex aspects of visual input require the vMMN. Our Gabor patches changed only one low-level property—orientation—without any other low-level changes, such as of contrast, luminance, spatial frequency, or retinal location. Other studies with spatially complex figures, such as windmill patterns, providing more global, higher-level properties of the stimuli, have generally found a genuine vMMN [37,71–73].
Precision of prediction.
The most likely tool in predictive-coding theory’s box is precision of prediction. This is the notion that variability of environment determines prediction precision, which, in turn, determines the prediction error [16,74]. Attention modulates precision [13,75]. Smout and colleagues found supporting evidence, admittedly only from their endogenously attended condition, by showing that the selectivity of a mismatch signal was greater when an unexpected orientation change followed more identical presentations of an unchanging orientation.
Implications for models of attentional control and stimulus selection
There are various neural models of attentional control and stimulus selection [76–78]. They share being brain-wide neural pathways, focusing on separate but interacting pathways from the sensory cortices to higher levels: the dorsal attention network and the ventral attention network [79,80]. The dorsal network, involving goal-directed, endogenous attention, includes sensory cortices for whatever modality is required to accomplish a goal, such as the visual cortex when searching for red fruit in green foliage. It has feedforward and feedback connections to numerous brain areas mainly of the dorsal pathways including dorsal frontoparietal regions involved with attentional control. The dorsal network is active for as long as the searching continues, maintaining goal-directed behavior including finding and picking the fruit, enhancing processing in the visual cortex for features specific to red fruit, and suppressing the ventral network.
The ventral network also begins with the visual cortex when an unattended, unexpected change in visual input happens, such as the appearance of a venomous snake in the fruit tree. The visual cortex has other feedforward and feedback connections to numerous brain areas mainly of the ventral pathways including middle and inferior prefrontal regions. Then, both networks become transiently activated to accomplish the new goal-directed behavior: to escape from the snake.
Ideally, the activity of the 2 networks correspond to the components typically found from ERP studies of changes in visual input. When an unexpected change is unattended, the ventral network is supposed to mediate the vMMN [81]; when the unexpected change is attended (task relevant), then the dorsal network is supposed to mediate the later, P300 component [82]. But the coarse resolution of source localization from ERP studies casts some doubt on whether this ideal is true [79].
The speculative implication of our finding of the EDRP is that sometimes an unattended, unexpected orientation change can fail to activate the supposed ventral network’s vMMN. To distinguish this notion from some explanation via low precision of prediction will require studies beyond the scope of ours, perhaps with advanced source resolution or using dynamic causal modeling.
Conclusion
We hope we have convinced you it is possible that unattended, unexpected changes in orientation do not yield the vMMN (consistent with the findings of Smout and colleagues) but might yield the EDRP. If so, the EDRP is a useful lead to understanding how the absence of a vMMN without attention could be reconciled with predictive-coding theory and models of attentional control and stimulus selection.
Methods
We showed 21 [Smout and colleagues had 24] participants sequences of identically oriented Gabor patches (Fig 1A), standards, and, unpredictably, otherwise identical Gabor patches differing in orientation by ±15°, ±30°, and ±60° [Smout and colleagues used ±20°, ±40°, ±60°, and ±80°; we continue to give their values in brackets after ours], deviants (80 ms duration [100 ms], 280 ms [500 ms] interstimulus interval, 634 patches [415] per block; see Fig 1A). Participants looked at a central, desynchronized stream of letters and pressed a key whenever an X appeared (Fig 1B). Standard sequences ranged in length randomly from 3 to more than 11. For an equiprobable control, we presented the same stimuli, but in a random order (see Fig 1D).
Ethics statement
The Human Research Ethics Committee at Murdoch University approved the experiment (permit 2015 208). All protocols adhered to the principles of the Declaration of Helsinki. All participants provided their written informed consent and were free to withdraw from the experiment at any time.
Participants
Twenty-one self-declared healthy adults (8 males, 18 right-handed) with normal or corrected-to-normal vision participated in the experiment (statistical power = .88, based on Kimura and Takeda’s [39] reported effect size for genuine vMMN at PO8: Cohen’s d = −0.73). The mean age was 33 years with a range from 18 to 60 years. Most of the participants were undergraduate psychology students at Murdoch University. Participants received course credit or entry into a draw to win a $50 gift card in return for participation.
Apparatus
Participants sat in a light-attenuated chamber viewing a calibrated monitor (17-inch, color, cathode ray tube display; Sony Trinitron Multiscan E230) from 57 cm. The monitor showed 1,280 × 1,024 pixels (75 Hz refresh rate) and was the only source of light. A chin rest stabilized each participant’s head. Participants gave their responses by pressing a key on a 4-key EGI response box with the index finger of their dominant hand.
A PC running Linux (v4.13.0), GNU Ubuntu (v16.04.4), Octave (v4.0.0) [83], and Psychophysics Toolbox (v3.0.14) [84–86] delivered the visual stimuli and recorded behavioral responses. An iMac running EGI’s NetStation 5.2 recorded EEG data.
Stimuli
We used achromatic Gabor patches (mean RGB values of 128, 127, 128) on an average gray background (RGB values of 128, 127, 128). A Gabor patch comprises a grating of a particular spatial frequency and orientation whose contrast reduces with distance from the center of the grating according to a Gaussian function [87,88]. These stimuli are like those used by Smout and colleagues.
Gabor patches had a contrast of .999 [Smout and colleagues used 1.00], a phase of 0 radians—the black-to-white crossing was in the center of the patch [not specified], a spatial frequency of 2.4 cycles per degree of visual angle [2.73], and a standard deviation of the Gaussian of 3.84° of visual angle [not specified]. The visible parts of each Gabor patch had a diameter of approximately 22° of visual angle [11°, with a 0.83° central blank area]. There were 12 possible orientations: 8°, 23°, 38°, 53°, 68°, 83°, 98°, 113°, 128°, 143°, 158°, and 173° clockwise from vertical (0°) [9 possible orientations from 0° to 160°, in 20° steps] (further details in S1 Table).
For the participants’ primary task, capitalized letters were superimposed on the center of the Gabor patch in cyan (RGB values of 0, 255, 255) in 30-point Courier font [a black fixation spot]. Letters occupied 0.5° (width) × 0.6° (height) with a line width of 0.06° [visual angle of 0.3° diameter]. We used all letters of the English alphabet except for “I”. We give an illustration in Fig 1.
Procedure
The participants’ task was to fixate on, and attend to, the continually changing random sequence of letters in the center of the monitor and to press a key with the dominant hand whenever an X appeared. Each letter occurred for 600 ms. None of the letters was immediately repeated. If the participant responded between 0.15 and 1.2 seconds after target onset, the response was correct. There were 383 letter changes during a block. On average, there were 15 targets (ranging from 7 to 25) per block.
Desynchronized from the letters, we presented our stimuli of interest—concentric Gabor patches—for 80 ms separated by a 280-ms interstimulus interval. We had 2 sorts of blocks.
Roving-standard blocks
The experimental, roving-standard blocks contained 634 trials [Smout and colleagues: 415] showing Gabor patches of which 114 (17.98%) [59 (14.29%)] were deviants. Standard trials showed orientations from among twelve 15° steps from 8° clockwise from vertical (0°) [nine 20° steps from 0°]. Deviant Gabor patches differed in orientation from the preceding standard stimuli by ±15°, ±30°, or ±60° [±20°, ±40°, ±60°, or ±80°].
We used a hazard distribution to equalize the probability of the appearance of a deviant on any trial. Consequently, the distribution of numbers of standard trials against serial position had a gamma shape, with fewest trials for the largest serial positions [89,90]. We give the mean percentages of the numbers of standard-trial runs across all participants in S1 Table. On average, 5.53 standards preceded each deviant. Orientation for each standard was approximately 8.3% each. We give the mean numbers over participants in S1 Table.
Equiprobable-control blocks
For each participant, we built equiprobable-control blocks from that participant’s roving-standard blocks. We replaced each block’s standard stimuli—except for the ones immediately preceding a deviant and the deviant itself—with a Gabor patch whose orientation was randomly chosen among the 12 possible orientations. The stimulus preceding the deviant and the deviant itself were identical in the 2 sorts of blocks. Control trials were the trials in the equiprobable-control block that were identical to the roving-standard block’s deviant trials. These 2 identical trials in both sorts of blocks allow us to discount the possibility that a difference in pairing of trials contributed to any deviant-related differences.
There were 6 roving-standard blocks and 6 equiprobable-control blocks. We randomized block order afresh for each participant. Each block took less than 4 minutes to complete. Participants were free to take breaks between blocks.
EEG recording and analysis
We recorded the electroencephalogram (EEG) using an EGI, 129-channel, dense-array HydroCel geodesic sensor net, and Net Amps 300 signal amplifier. We recorded EEG at a 500-Hz sampling rate. Impedances were below 50 kΩ, per recommendation [91] and existing standard (e.g., [72,92–95]) for high-input impedance amplifiers. All electrodes were referenced to Cz.
We processed the EEG data offline using MATLAB 2015b (MathWorks, USA), EEGLAB 14.1.1 [96], and ERPLAB 6.1.4 [97]. We re-referenced the signal of all electrodes to the common average and filtered the EEG with a low-pass 40-Hz Kaiser-windowed (beta 5.65) sinc finite impulse response (FIR) filter (order 184) followed by a high-pass 0.1-Hz Kaiser-windowed (beta 5.65) sinc FIR filter (order 9056). Epochs were 400 ms long, featuring a 50-ms prestimulus period for baseline correction to accommodate the short 360 ms stimulus-onset-asynchrony. We excluded epochs with amplitude changes exceeding 800 μV at any electrode [98,99].
We identified electrodes with unusually high deviations in EEG activity relative to the average standard deviation pooled from all electrodes using the method described by Bigdely-Shamlo and colleagues [100]. A robust z-score was calculated for each electrode by replacing the mean by the median and the standard deviation by the robust standard deviation (0.7413 times the interquartile range). We removed any electrode with a robust z-score exceeding 2.0 provided at least 4 others surrounded it for later interpolation.
We created vertical and horizontal EOG channels by bipolarizing data from electrodes above and below the right eye (electrodes 8 and 126) and outer canthi of both eyes (electrodes 1 and 32), respectively (as in [72]). To ensure that trials where participants moved their eyes or blinked were not included in the final analysis of the data, we identified epochs containing amplitude changes exceeding ±60 μV at these EOG channels for rejection prior to ICA correction. We did not exclude these epochs at this stage, and ICA artifact correction had not yet occurred.
We performed independent component analysis (ICA) with AMICA [101]. To improve the decomposition, we performed the analysis on raw data (excluding bad electrodes) that was first filtered using a 1-Hz high-pass (Kaiser-windowed sinc FIR filter, order 804, beta 5.65) and 40-Hz low-pass filter as suggested by Winkler and colleagues [98]. Thereafter, we segmented into epochs, but without baseline correction [102]. We simultaneously reduced the data to 32 components.
We applied the demixing matrix to the 0.1- to 40-Hz filtered data and used SASICA [103] to identify which components exhibited low autocorrelation, low focal electrode or trial activity, high correlation with vertical or horizontal EOG, or met ADJUST criteria [104]. We assessed the remaining components using criteria described by Chaumon and colleagues [105], classifying components based on consistent activity time-locked to stimulus onset across all trials, on topography, or on power spectrum. We removed components identified as unrelated to brain activity.
We then removed epochs that were previously identified for rejection. This was followed by a final artifact rejection—removing epochs containing amplitude changes exceeding ±60 μV at any electrode. Finally, using spherical splines [106], we interpolated data for removed electrodes.
For exploring deviant-related responses, we averaged ERPs for the standard, deviant, and control trials, excluding epochs immediately following a deviant or control trial. We produced difference waves by subtracting ERPs to standards and ERPs to controls from ERPs to deviants.
For exploring repetition suppression, we averaged PCA components for standards from roving-standard blocks and for randomly oriented stimuli from equiprobable-control blocks at varying sequential positions, excluding deviant and control trials. We retained standard trials immediately following deviants; we retained randomly oriented trials immediately following controls (S1 Table for number of epochs in each ERP).
We defined anterior, central, and PO regions of interest (ROIs). The signal from all electrodes within each ROI was averaged over time. We visually inspected ERPs for electrodes within each ROI and then when averaged together. There was middle (M) frontal (M F) central (M C), parieto-occipital (M PO), left (L) frontal (L F), central (M C), parieto-occipital (L PO), and right frontal (R F), central (R C), and parieto-occipital (R PO). We display the results of 6 ROIs (M F, M C, M PO, L PO, R PO; see S2 Table for electrodes within each cluster) and 1 PZ electrode for comparison with Smout and colleagues.
Using the EP Toolkit (v2.64; [34]), we conducted temporal PCA on the individual average ERP data from all 129 electrodes, thereby allowing for deviant-related responses isolated to any one electrode to emerge. For exploring deviant-related responses, we included deviant and control trials. For exploring repetition suppression, we included standard and random trials. We used Promax orthogonal rotation (κ = 3) with a covariance relationship matrix and Kaiser weighting as recommended by Dien and colleagues [107]. PCA reduced the data to those components explaining most of the observed data. The component that explained the greatest amount of signal is component 1, and every component thereafter explained less of the data than the component before it. We show how much of the signal a component is responsible for by plotting the component’s loading (scaled by SD) over time [108].
Using PCA, one can identify separate components in the ERP waveform and extract an alternative measure of ERP component amplitudes for inferential testing [107,109,110]. Each PCA component has a peak latency, a site of maximum positivity on the scalp (i.e., a component’s positive pole), and a site of maximum negativity on the scalp (i.e., a component’s negative pole). Plotting a topographical map of the microvolt scaled PCA data of a single component at the time of its peak latency shows the component’s positive and negative pole.
A vMMN component would emerge as a component that is largest (most negative) between 150 and 300 ms, based on previous vMMN studies using orientation deviants (e.g., 162 to 170 ms in [43]; 200 to 250 ms in [22]; 190 to 220 ms in [39]). A genuine vMMN component should also yield scores that are more negative for deviants compared to controls at the component’s negative pole.
In addition to our frequentist paired t tests, we conducted Bayesian paired t tests to determine the likelihood of obtaining the data. We used a medium prior (with a Cauchy prior whose width was set to 0.707) for all Bayesian analyses. For interpreting our findings, a model with the largest Bayes Factor (BF10) is the model that best explains the data; this is the favored model. All main effects and interactions in the favored model are, therefore, important for explaining the data. Evidence against the null is considered weak if a BF10 is between 1 and 3. It is positive for a BF10 between 3 and 20, strong for a BF10 between 20 and 150, and very strong given a BF10 greater than 150 [111].
We also performed Bayes Factor replication (BFr0) tests [112] on mean amplitudes between 197 and 207 ms at each PO ROI given a prior reflecting the effect size (Cohen’s d = −0.73) for the genuine vMMN at the PO8 in Kimura and Takeda’s [39] oddball condition for 32.7° orientation deviants.
For all frequentist ANOVAs, we applied the Greenhouse–Geisser correction where necessary (ε < .750). Eta squared (η2) denotes the estimated effect size.
Supporting information
S1 Fig.
(A) Grand average ERPs for 15° orientation changes from various electrode clusters around the electrodes Smout and colleagues [24] reported. (B) Difference waves for classic (15-degree deviant minus standard) and genuine (15-degree deviant minus control) deviant-related activity. The arrowed components show the only genuine EDRP. Horizontal gray bars illustrate the time-window in which Kimura and Takeda [39] found the largest deviant-minus-control difference (i.e., genuine vMMN) for 32.7° orientation deviants. Mean amplitudes from this time window were analyzed using Bayesian replication (results in S3 Table). The lighter colors surrounding the difference waves give ±1 standard error of the mean (data in S3 Data). EDRP, early deviant-related positivity; ERP, event-related potential; vMMN, visual mismatch negativity.
https://doi.org/10.1371/journal.pbio.3001866.s001
(PDF)
S2 Fig.
(A) Grand average ERPs for 30° orientation changes from various electrode clusters around the electrodes Smout and colleagues [24] reported. (B) Difference waves for classic (30-degree deviant minus standard) and genuine (30-degree deviant minus control) deviant-related activity. The arrowed components show the only genuine EDRP. Horizontal gray bars illustrate the time-window in which Kimura and Takeda [39] found the largest deviant-minus-control difference (i.e., genuine vMMN) for 32.7° orientation deviants. Mean amplitudes from this time window were analyzed using Bayesian replication (results in S3 Table). The lighter colors surrounding the difference waves give ±1 standard error of the mean (data in S3 Data). EDRP, early deviant-related positivity; ERP, event-related potential; vMMN, visual mismatch negativity.
https://doi.org/10.1371/journal.pbio.3001866.s002
(PDF)
S3 Fig. N1 repetition suppression results.
N1 PCA scores are shown separately for the left (L), midline (M), and right (R) PO ROIs. Regression equations show the dotted lines in the data for standard and random stimuli. Asterisks denote the linear regression significance: *p < .05, **p < .01, ***p < .001. Error bars show 1 SE. We show −1 SE for standard and +1 SE for random stimuli to avoid overlap (data in S5 Data). PCA, principal component analysis; PO, parieto-occipital; ROI, region of interest; SE, standard error.
https://doi.org/10.1371/journal.pbio.3001866.s003
(PDF)
S1 Data.
The number of trials, targets, targets correctly identified (Hit N), targets not identified (Miss N), targets incorrectly identified (False Alarm N), mean reaction time (RT), hit rate, and false alarm rate for Roving and Equiprobable sequences appear for each participant on separate rows. Means and standard deviations for Roving and Equiprobable sequences appear in the last 2 rows. Data summary and analysis results in S1 Supporting Information—Behavioral Results.
https://doi.org/10.1371/journal.pbio.3001866.s004
(XLSX)
S2 Data. Mean amplitudes and difference wave mean amplitudes (μv) at left (L), middle (M), and right (R) parieto-occipital (PO) regions between 197 and 207 ms for each magnitude of deviance (Small 15-degree Deviant, Medium 30-degree Deviant, Large 60-degree Deviant) and condition (Standards, Deviants, Controls, Deviant-minus-Standard, and Deviant-minus-Control) for each participant on separate rows.
Data summary and analysis results in S3 Table.
https://doi.org/10.1371/journal.pbio.3001866.s005
(XLSX)
S3 Data. Amplitudes (μv) over time (−50 to 348 ms) at middle (M) frontal (M F), middle central (M C), middle parieto-occipital (M PO), left parieto-occipital (L PO), right parieto-occipital (R PO), and parietal (PZ) regions for each magnitude of deviance (Small 15-degree Deviant [S1 Fig], Medium 30-degree Deviant [S2 Fig], Large 60-degree Deviant [Fig 2]) and condition (Standards, Deviants, Controls, Deviant-minus-Standard, and Deviant-minus-Control) for each participant on separate rows.
https://doi.org/10.1371/journal.pbio.3001866.s006
(XLSX)
S4 Data. P1 and N1 principal component scores at left parieto-occipital (L PO), middle parieto-occipital (M PO), and right parieto-occipital (R PO) regions for each magnitude of deviance (Small 15-degree Deviant, Medium 30-degree Deviant, Large 60-degree Deviant) and condition (Standards, Deviants, Controls, Deviant-minus-Standard, and Deviant-minus-Control) for each participant on separate rows.
Data summary and analysis results in S1 Supporting Information—Deviant-related positivity and Fig 3.
https://doi.org/10.1371/journal.pbio.3001866.s007
(XLSX)
S5 Data. Means and standard errors for P2 and N1 principal component scores at left parieto-occipital (L PO), middle parieto-occipital (M PO), and right parieto-occipital (R PO) regions for standard and random stimuli at positions: 2, 3, 4, 5, 6, 7, 8, and 10.36 (the average number of ≥9 standards).
Data summary and analysis results in S1 Supporting Information—Repetition suppression and Figs 4 (P2) and S3 (N1).
https://doi.org/10.1371/journal.pbio.3001866.s008
(XLSX)
S1 Table. Orientation probabilities for standard and random stimuli appearing in roving-standard and equiprobable-control blocks, the number of standard repetitions, and the mean number of epochs (and standard deviations) for each event-related potential (ERP).
https://doi.org/10.1371/journal.pbio.3001866.s009
(DOCX)
S2 Table. Electrode numbers on 128-channel geodesic sensor net within left, middle, right, frontal, parieto-occipital, and central regions of interest (ROIs) with corresponding 10–20 electrodes.
https://doi.org/10.1371/journal.pbio.3001866.s010
(DOCX)
S3 Table. Statistical analysis of the difference wave mean amplitudes at the left, middle, and right parieto-occipital regions of interest (ROIs) between 197 and 207 ms for each magnitude of deviance (Small 15-degree Deviant, Medium 30-degree Deviant, Large 60-degree Deviant).
https://doi.org/10.1371/journal.pbio.3001866.s011
(DOCX)
S1 Supporting Information. Results of statistical analysis of behavioral (S1) and electrophysiological (S2) data presented in the main text.
Behavioral results include hit rates, false alarm rates, and reaction times. Electrophysiological data include deviant-related responses and repetition suppression.
https://doi.org/10.1371/journal.pbio.3001866.s012
(DOCX)
Acknowledgments
We are grateful to Urte Roeber, who contributed to conception, methodology, review, and supervision; to Sabine Grimm, Cooper Smout, and Philip Smith for helpful discussion; and to the students who assisted in collecting the data for this experiment.
References
- 1. Attneave F. Some informational aspects of visual perception. Psychol Rev. 1954;61:183–93. pmid:13167245
- 2.
James W. The principles of psychology. New York: Henry Holt; 1890.
- 3. Srinivasan MV, Laughlin SB, Dubs A. Predictive coding: A fresh view of inhibition in the retina. Proc R Soc Lond B Biol Sci. 1982;216:427–59. pmid:6129637
- 4. Bastos AM, Usrey WM, Adams RA, Mangun GR, Fries P, Friston KJ. Canonical microcircuits for predictive coding. Neuron. 2012;76:695–711. Available from: org.prospero.murdoch.edu.au/10.1016/j.neuron.2012.10.038. pmid:23177956
- 5. Rao RPN, Ballard DH. Dynamic model of visual recognition predicts neural response properties in the visual cortex. Neural Comput. 1997;9(4):721–763. pmid:9161021
- 6. Huang Y, Rao RPN. Predictive coding. Wiley Interdiscip Rev Cogn Sci. 2011;2(5):580–93. pmid:26302308
- 7. Friston K. The free-energy principle: a unified brain theory? Nat Rev Neurosci. 2010;11(2):127–138. pmid:20068583
- 8. Friston K. Learning and inference in the brain. Neural Netw. 2003;16(9):1325–1352. pmid:14622888
- 9. Garrido MI, Friston KJ, Kiebel SJ, Stephan KE, Baldeweg T, Kilner JM. The functional anatomy of the MMN: A DCM study of the roving paradigm. Neuroimage. 2008;42(2):936–944. 1506754467. pmid:18602841
- 10. Garrido MI, Kilner JM, Stephan KE, Friston KJ. The mismatch negativity: A review of underlying mechanisms. Clin. Neurophysiol. 2009;120(3):453–463. pmid:19181570
- 11. Clark A. Whatever next? Predictive brains, situated agents, and the future of cognitive science. Behav Brain Sci. 2013;36(3):181–204. 1415798529; pmid:23663408
- 12. Spratling MW. A review of predictive coding algorithms. Brain Cogn. 2017;112:92–97. pmid:26809759
- 13. Friston KJ, Kiebel S. Cortical circuits for perceptual inference. Neural Netw. 2009;22(8):1093–1104. pmid:19635656
- 14. Friston KJ, Mattout J, Trujillo-Barreto N, Ashburner J, Penny W. Variational free energy and the Laplace approximation. Neuroimage. 2007;34(1):220–234. pmid:17055746
- 15. Friston KJ, Kilner J, Harrison L. A free energy principle for the brain. J Physiol Paris. 2006;100(1–3):70–87. pmid:17097864
- 16. Yeark M, Paton B, Todd J. The influence of variability on mismatch negativity amplitude. Biol Psychol. 2021;164. pmid:34333068
- 17. Yeark M, Paton B, Brown A, Raal A, Todd J. Primacy biases endure the addition of frequency variability. Neuropsychologia. 2022;171:108233. pmid:35500824
- 18. Feldman H, Friston KJ. Attention, Uncertainty, and Free-Energy. Front Hum Neurosci. 2010;4. pmid:21160551
- 19.
Rao RPN, Ballard DH. CHAPTER 91—Probabilistic Models of Attention Based on Iconic Representations and Predictive Coding. In: Itti L, Rees G, Tsotsos JK, editors. Neurobiology of Attention. Burlington: Academic Press; 2005. p. 553–561.
- 20. Stefanics G, Kremláček J, Czigler I. Visual mismatch negativity: A predictive coding view. Front Hum Neurosci. 2014;8(666):1–19. pmid:25278859
- 21. Schröger E, Wolff C. Mismatch response of the human brain to changes in sound location. Neuroreport. 1996;7(18):3005–3008. pmid:9116228
- 22. Kimura M, Katayama J, Ohira H, Schröger E. Visual mismatch negativity: new evidence from the equiprobable paradigm. Psychophysiology. 2009;46(2):402–9. pmid:19207207
- 23. Paavilainen P, Alho K, Reinikainen K, Sams M, Näätänen R. Right hemisphere dominance of different mismatch negativities. Electroencephalogr Clin Neurophysiol. 1991;78:466–479. pmid:1712282
- 24. Smout CA, Tang MF, Garrido MI, Mattingley JB. Attention promotes the neural encoding of prediction errors. PLoS Biol. 2019;17(2):1–22. https://dx.doi.org/10.1371/journal.pbio.2006812
- 25. Bendixen A, Roeber U, Schröger E. Regularity extraction and application in dynamic auditory stimulus sequences. J Cogn Neurosci. 2007;19:1664–1677. pmid:18271740
- 26. Hubel DH, Wiesel TN. Receptive fields and functional architecture of monkey striate cortex. J Physiol. 1968;195:215–243. pmid:4966457
- 27. Auksztulewicz R, Friston K. Attentional enhancement of auditory mismatch responses: A DCM/MEG study. Cereb Cortex. 2015;25(11):4273–4283. pmid:25596591
- 28. Schwartz EL. Computational anatomy and functional architecture of striate cortex: A spatial mapping approach to perceptual coding. Vision Res. 1980;20(8):645–669. pmid:7445436
- 29. Male AG , O’Shea RP, Schröger E, Müller D, Roeber U, Widmann A. The quest for the genuine visual mismatch negativity (vMMN): Event-related potential indications of deviance detection for low-level visual features. Psychophysiology. 2020;1(1):1–10. pmid:32293040
- 30. Scrivener CL, Reader AT. Variability of EEG electrode positions and their underlying brain regions: Visualizing gel artifacts from a simultaneous EEG-fMRI dataset. Brain Behav. 2022;12(2):e2476. pmid:35040596
- 31. Stefanics G, Heinzle J, Czigler I, Valentini E, Stephan KE. Timing of repetition suppression of event-related potentials to unattended objects. Eur J Neurosci. 2020;52(11):4432–4441. pmid:29802671
- 32. Grill-Spector K, Henson R, Martin A. Repetition and the brain: neural models of stimulus-specific effects. Trends Cogn Sci. 2006;10(1):14–23. pmid:16321563
- 33. Tang MF, Smout CA, Arabzadeh E, Mattingley JB. Prediction error and repetition suppression have distinct effects on neural representations of visual information. Elife. 2018;7:1–21. pmid:30547881
- 34. Dien J, Khoe W, Mangun GR. Evaluation of PCA and ICA of simulated ERPs: Promax vs. infomax rotations. Hum Brain Mapp. 2007;28(8):742–763. pmid:17133395
- 35. Schendan HE, Kutas M. Time course of processes and representations supporting visual object identification and memory. J Cogn Neurosci. 2003;15(1):111–135. pmid:12590847
- 36. Henson RN, Rylands A, Ross E, Vuilleumeir P, Rugg MD. The effect of repetition lag on electrophysiological and haemodynamic correlates of visual object priming. Neuroimage. 2004;21(4):1674–1689. pmid:15050590
- 37. File D, File B, Bodnár F, Sulykos I, Kecskés-Kovács K, Czigler I. Visual mismatch negativity (vMMN) for low- and high-level deviances: A control study. Atten Percept Psychophys. 2017;79(7):2153–2170. pmid:28710556
- 38. Sulykos I, Kecskés-Kovács K, Czigler I. Mismatch Negativity Does Not Show Evidence of Memory Reactivation in the Visual Modality. J Psychophysiol. 2013;27(1):1–6. https://dx.doi.org/10.1027/0269-8803/a000085
- 39. Kimura M, Takeda Y. Automatic prediction regarding the next state of a visual object: Electrophysiological indicators of prediction match and mismatch. Brain Res. 2015:31–44. pmid:25598206.
- 40. Astikainen P, Ruusuvirta T, Wikgren J, Korhonen T. The human brain processes visual changes that are not cued by attended auditory stimulation. Neurosci Lett. 2004;368(2):231–234. pmid:15351455.
- 41. Kimura M, Takeda Y. Task difficulty affects the predictive process indexed by visual mismatch negativity. Front Hum Neurosci. 2013;7(267):1–13. pmid:23781189
- 42. Burbeck CA, Regan D. Independence of orientation and size in spatial discriminations. J Opt Soc Am. 1983;73(12):1691–1694. Epub 1983/12/01. pmid:6663372.
- 43. Czigler I, Sulykos I. Visual mismatch negativity to irrelevant changes is sensitive to task-relevant changes. Neuropsychologia. 2010;48(5):1277–1282. pmid:20036268.
- 44. Astikainen P, Lillstrang E, Ruusuvirta T. Visual mismatch negativity for changes in orientation—a sensory memory-dependent response. Eur J Neurosci. 2008;28(11):2319–2324. pmid:19019200.
- 45. Pesonen H, Savić AM, Kujala UM, Tarkka IM. Long-term physical activity modifies automatic visual processing. Int J Sport Exerc Psychol. 2017;2017:1–10. https://dx.doi.org/10.1080/1612197X.2017.1321031
- 46. Czigler I, Balázs L, Winkler I. Memory-based detection of task-irrelevant visual changes. Psychophysiology. 2002;39(6):869–873. pmid:12462515
- 47. Schmitt C, Klingenhoefer S, Bremmer F. Preattentive and predictive processing of visual motion. Sci Rep. 2018;8(12399):1–12. pmid:30120337
- 48. Czigler I, Pató L. Unnoticed regularity violation elicits change-related brain activity. Biol Psychol. 2009;80(3):339–347. pmid:19111891
- 49. Fisher DJ, Scott TL, Shah DK, Prise S, Thompson M, Knott VJ. Light up and see: Enhancement of the visual mismatch negativity (vMMN) by nicotine. Brain Res. 2009;1313:162–171. pmid:20006592
- 50. Leung S, Greenwood L, Michie P, Croft R. The role of stimulus train length in mismatch negativity (MMN) abnormalities in schizophrenia: A comparison of the ’roving’ and ’oddball’ MMN paradigms. Front Hum Neurosci. 2015;9. https://dx.doi.org/10.3389/conf.fnhum.2015.217.00153
- 51. Cowan N, Winkler I, Teder W, Näätänen R. Memory prerequisites of mismatch negativity in the auditory event-related potential (ERP). J Exp Psychol Learn Mem Cogn. 1993;19(4):909–921. pmid:8345328
- 52. Lamme VA, Roelfsema PR. The distinct modes of vision offered by feedforward and recurrent processing. Trends Neurosci. 2000;23(11):571–579. Epub 2000/11/14. pmid:11074267.
- 53. Dehaene S, Changeux J. Experimental and Theoretical Approaches to Conscious Processing. Neuron. 2011;70(2):200–227. pmid:21521609
- 54. Noah S, Meyyappan S, Ding M, Mangun GR. Time courses of attended and ignored object representations. 2022. https://doi.org/10.31234/osf.io/2aj3n
- 55. Grootswagers T, Robinson AK, Shatek SM, Carlson TA. The neural dynamics underlying prioritisation of task-relevant information. arXiv preprint arXiv:210201303. 2021.
- 56. Moerel D, Grootswagers T, Robinson AK, Shatek SM, Woolgar A, Carlson TA, et al. The time-course of feature-based attention effects dissociated from temporal expectation and target-related processes. Sci Rep. 2022;12(1):6968. pmid:35484363
- 57. Chen Y, Huang X, Luo Y, Peng C, Liu C. Differences in the neural basis of automatic auditory and visual time perception: ERP evidence from an across-modal delayed response oddball task. Brain Res. 2010;1325:100–111. pmid:20170647
- 58. Fu S, Fan S, Chen L. Event-related potentials reveal involuntary processing of orientation changes in the visual modality. Psychophysiology. 2003;40(5):770–775. pmid:14696730
- 59. Sysoeva OV, Lange EB, Sorokin AB, Campbell T. From pre-attentive processes to durable representation: An ERP index of visual distraction. Int J Psychophysiol. 2015;95(3):310–321. pmid:25523346
- 60. Müller D, Roeber U, Winkler I, Trujillo-Barreto N, Czigler I, Schröger E. Impact of lower- vs. upper-hemifield presentation on automatic colour-deviance detection: A visual mismatch negativity study. Brain Res. 2012;1472:89–98. pmid:22820304
- 61. Berti S, Schröger E. Distraction effects in vision: behavioral and event-related potential indices. Neuroreport. 2004;15(4):665–669. pmid:15094472
- 62. Näätänen R, Paavilainen P, Rinne T, Alho K. The mismatch negativity (MMN) in basic research of central auditory processing: A review. Clin Neurophysiol. 2007;118(12):2544–2590. pmid:17931964.
- 63. O’Reilly JA , O’Reilly A. A critical review of the deviance detection theory of mismatch negativity. NeuroSci. 2021;2(2):151–165.
- 64. Näätänen R, Gaillard AWK, Mäntysalo S. Early selective-attention effect on evoked potential reinterpreted. Acta Psychol (Amst). 1978;42(4):313–329. pmid:685709
- 65. Cornella M, Leung S, Grimm S, Escera C. Detection of simple and pattern regularity violations occurs at different levels of the auditory hierarchy. PLoS ONE. 2012;7(8):e43604. pmid:22916282
- 66. Cornella M, Leung S, Grimm S, Escera C. Regularity encoding and deviance detection of frequency modulated sweeps: Human middle- and long-latency auditory evoked potentials. Psychophysiology. 2013;50(12):1275–1281. pmid:24016075
- 67. Escera C, Leung S, Grimm S. Deviance detection based on regularity encoding along the auditory hierarchy: Electrophysiological evidence in humans. Brain Topogr. 2014;27(4):527–538. pmid:24218032
- 68. Leung S, Recasens M, Grimm S, Escera C. Electrophysiological index of acoustic temporal regularity violation in the middle latency range. Clin. Neurophysiol. 2013;124(12):2397–2405. pmid:23830004
- 69. Grimm S, Escera C, Slabu L, Costa-Faidella J. Electrophysiological evidence for the hierarchical organization of auditory change detection in the human brain. Psychophysiology. 2011;48(3):377–384. pmid:20636288
- 70. Grimm S, Recasens M, Althen H, Escera C. Ultrafast tracking of sound location changes as revealed by human auditory evoked potentials. Biol Psychol. 2012;89(1):232–9. Epub 2011/11/01. pmid:22037091.
- 71. Maekawa T, Goto Y, Kinukawa N, Taniwaki T, Kanba S, Tobimatsu S. Functional characterization of mismatch negativity to a visual stimulus. Clin Neurophysiol. 2005;116(10):2392–2402. pmid:16122977.
- 72. Maekawa T, Katsuki S, Kishimoto J, Onitsuka T, Ogata K, Yamasaki T, et al. Altered visual information processing systems in bipolar disorder: evidence from visual MMN and P3. Front Hum Neurosci. 2013;7:403. Epub 2013/07/31. pmid:23898256; PubMed Central PMCID: PMC3724050.
- 73. Maekawa T, Tobimatsu S, Ogata K, Onitsuka T, Kanba S. Preattentive visual change detection as reflected by the mismatch negativity (MMN): Evidence for a memory-based process. Neurosci Res. 2009;65(1):107–112. pmid:19523998.
- 74. Fitzgerald K, Todd J. Making sense of mismatch negativity. Front Psych. 2020;11:468. Epub 2020/07/01. pmid:32595529; PubMed Central PMCID: PMC7300203.
- 75. Clark A., Predictions precision, and agentive attention. Conscious Cogn. 2017;56:115–119. pmid:28693812
- 76. Desimone R, Wessinger M, Thomas L, Schneider W. Attentional control of visual perception: cortical and subcortical mechanisms. Cold Spring Harb Symp Quant Biol. 1990;55:963–971. Epub 1990/01/01. pmid:2132873.
- 77. Hopfinger JB, Buonocore MH, Mangun GR. The neural mechanisms of top-down attentional control. Nat Neurosci. 2000;3(3):284–291. Epub 2000/03/04. pmid:10700262.
- 78. Eimer M. The neural basis of attentional control in visual search. Trends Cogn Sci. 2014;18(10):526–535. Epub 2014/06/16. pmid:24930047.
- 79. Corbetta M, Patel G, Shulman GL. The Reorienting System of the Human Brain: From Environment to Theory of Mind. Neuron. 2008;58(3):306–324. pmid:18466742
- 80. Corbetta M, Shulman GL. Control of goal-directed and stimulus-driven attention in the brain. Nat Rev Neurosci. 2002;3(3):201–215. pmid:11994752
- 81. Hopfinger JB, Ries AJ. Automatic versus contingent mechanisms of sensory-driven neural biasing and reflexive attention. J Cogn Neurosci. 2005;17(8):1341–1352. pmid:16197688
- 82. Mangun GR, Hillyard SA. Modulations of sensory-evoked brain potentials indicate changes in perceptual processing during visual-spatial priming. J Exp Psychol Hum Percept Perform. 1991;17(4):1057–1074. Epub 1991/11/01. pmid:1837297.
- 83.
Eaton JW, Bateman D, Hauberg S, Wehbring R. GNU Octave version 3.8.1 manual: A high-level interactive language for numerical computations.: CreateSpace Independent Publishing Platform; 2014.
- 84. Brainard DH. The Psychophysics Toolbox. Spat Vis. 1997;10(4):433–436. pmid:9176952
- 85. Kleiner M, Brainard D, Pelli D, Ingling A, Murray R, Broussard C, editors. What’s new in psychtoolbox-3. Proceedings of European Conference of Visual Perception; 2007: Perception.
- 86. Pelli DG. The VideoToolbox software for visual psychophysics: Transforming numbers into movies. Spat Vis. 1997;10(4):437–442. pmid:9176953
- 87.
Bruce V, Green PR, Georgeson MA. Visual perception: physiology, psychology, & ecology. 4th ed. Hove;New York: Psychology Press; 2003.
- 88. Fredericksen RE, Bex PJ, Verstraten FAJ. How big is a Gabor patch, and why should we care? errata. J Opt Soc Am A Opt Image Sci Vis. 1998;15(7):1. https://dx.doi.org/10.1364/JOSAA.15.001959
- 89.
Luce RD. Response Times: Their Role in Inferring Elementary Mental Organization: Oxford University Press; 1991 01 Jan 2008.
- 90. Chechile RA. Mathematical tools for hazard function analysis. J Math Psychol. 2003;47(5):478–494. https://doi.org/10.1016/S0022-2496(03)00063-4
- 91. Ferree TC, Luu P, Russell GS, Tucker DM. Scalp electrode impedance, infection risk, and EEG data quality. Clin Neurophysiol. 2001;112:538–544. pmid:11222977
- 92. Yu YH, Shafer VL. Neural representation of the English vowel feature [high]: Evidence from /ε/ vs. /ɪ. Front Hum Neurosci. 2021;15. https://dx.doi.org/10.3389/fnhum.2021.629517
- 93. Vilhelmsen K, van der Weel FR, van der Meer AL. A high-density EEG study of differences between three high speeds of simulated forward motion from optic flow in adult participants. Front Syst Neurosci. 2015;9:146. Epub 2015/11/19. pmid:26578903; PubMed Central PMCID: PMC4620151.
- 94. Gajos A, Wójcik GM. Independent component analysis of EEG data for EGI system. 2016;12(2):67–72.
- 95. Cai H, Yuan Z, Gao Y, Sun S, Li N, Tian F, et al. A multi-modal open dataset for mental-disorder analysis. Sci Data. 2022;9(1):178. pmid:35440583
- 96. Delorme A, Makeig S. EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J Neurosci Methods. 2004;134(1):9–21. pmid:15102499
- 97. Lopez-Calderon J, Luck SJ. ERPLAB: an open-source toolbox for the analysis of event-related potentials. Front Hum Neurosci. 2014;8(213). pmid:24782741
- 98. Winkler I, Debener S, Müller KR, Tangermann M. On the influence of high-pass filtering on ICA-based artifact reduction in EEG-ERP. Annu Int Conf IEEE Eng Med Biol Soc. 2015;2015:4101–4105. Epub 2016/01/07. pmid:26737196.
- 99. Dimigen O. Optimizing the ICA-based removal of ocular EEG artifacts from free viewing experiments. Neuroimage. 2020;207:116117. pmid:31689537
- 100. Bigdely-Shamlo N, Mullen T, Kothe C, Su K, Robbins KA. The PREP pipeline: Standardized preprocessing for large-scale EEG analysis. Front Neuroinform. 2015;9(16):1–20. pmid:26150785.
- 101. Palmer J, Kreutz-Delgado K. Makeig S. AMICA: An Adaptive Mixture of Independent Component Analyzers with Shared Components. Available from: https://dsp.ucsd.edu/~kreutz/Publications/palmer2011AMICA.pdf. San Diego: 2011.
- 102. Groppe DM, Makeig S, Kutas M. Identifying reliable independent components via split-half comparisons. Neuroimage. 2009;45(4):1199–1211. Epub 12/31. pmid:19162199.
- 103.
Makeig S, Bell AJ, Jung TP, Sejnowski TJ. Independent component analysis of electroencephalographic data. In: Touretzky D, Mozer M, Hasselmo M, editors. Advances in Neural Information Processing Systems. Cambridge: MIT Press; 1996.
- 104. Mognon A, Jovicich J, Bruzzone L, Buiatti M. ADJUST: An automatic EEG artifact detector based on the joint use of spatial and temporal features. Psychophysiology. 2011;48(2):229–240. pmid:20636297
- 105. Chaumon M, Bishop DVM, Busch NA. A practical guide to the selection of independent components of the electroencephalogram for artifact correction. J Neurosci Methods. 2015;250(Supplement C):47–63. https://doi.org/10.1016/j.jneumeth.2015.02.025
- 106. Perrin F, Pernier J, Bertnard O, Giard MH, Echallier JF. Mapping of scalp potentials by surface spline interpolation. Electroencephalogr Clin Neurophysiol. 1987;66(1):75–81. pmid:2431869
- 107.
Dien J, Frishkoff G. Introduction to Principal Components Analysis of Event-Related Potentials. In: Handy T, editor. Event Related Potentials: A Methods Handbook. Cambridge, Massachussetts: MIT Press; 2005.
- 108. Dien J. Applying principal components analysis to event-related potentials: a tutorial. Dev Neuropsychol. 2012;37(6):497–517. Epub 2012/08/15. pmid:22889342.
- 109. Carretié L, Tapia M, Mercado F, Albert J, López-Martín S, de la Serna JM. Voltage-based versus factor score-based source localization analyses of electrophysiological brain activity: A comparison. Brain Topogr. 2004;17(2):109–115. pmid:15754876
- 110. Dien J. The ERP PCA Toolkit: An open source program for advanced statistical analysis of event-related potential data. J Neurosci Methods 2010;187(1):138–145. pmid:20035787
- 111.
Raftery AE. Bayesian model selection in social research. In: Marsden PV, editor. Sociological methodology. Cambridge, MA: Blackwell; 1995. p. 111–196.
- 112. Verhagen J, Wagenmakers EJ. Bayesian tests to quantify the result of a replication attempt. J Exp Psychol Gen. 2014;143(4):1457–1475. pmid:24867486