http://orcid.org/0000-0001-9581-926X
Figures
Abstract
To understand information representation in human brain activity, it is important to investigate its fine spatial patterns at high temporal resolution. One possible approach is to use source estimation of magnetoencephalography (MEG) signals. Previous studies have mainly quantified accuracy of this technique according to positional deviations and dispersion of estimated sources, but it remains unclear how accurately MEG source estimation restores information content represented by spatial patterns of brain activity. In this study, using simulated MEG signals representing artificial experimental conditions, we performed MEG source estimation and multivariate pattern analysis to examine whether MEG source estimation can restore information content represented by patterns of cortical current in source brain areas. Classification analysis revealed that the corresponding artificial experimental conditions were predicted accurately from patterns of cortical current estimated in the source brain areas. However, accurate predictions were also possible from brain areas whose original sources were not defined. Searchlight decoding further revealed that this unexpected prediction was possible across wide brain areas beyond the original source locations, indicating that information contained in the original sources can spread through MEG source estimation. This phenomenon of “information spreading” may easily lead to false-positive interpretations when MEG source estimation and classification analysis are combined to identify brain areas that represent target information. Real MEG data analyses also showed that presented stimuli were able to be predicted in the higher visual cortex at the same latency as in the primary visual cortex, also suggesting that information spreading took place. These results indicate that careful inspection is necessary to avoid false-positive interpretations when MEG source estimation and multivariate pattern analysis are combined.
Citation: Sato M, Yamashita O, Sato M-a, Miyawaki Y (2018) Information spreading by a combination of MEG source estimation and multivariate pattern classification. PLoS ONE 13(6): e0198806. https://doi.org/10.1371/journal.pone.0198806
Editor: Christos Papadelis, Boston Children's Hospital / Harvard Medical School, UNITED STATES
Received: October 18, 2017; Accepted: May 27, 2018; Published: June 18, 2018
Copyright: © 2018 Sato et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: This work was supported by JSPS KAKENHI (JP16H01541, JP17H01755 and JP26120514), MIC SCOPE (141203025), JST PRESTO (JPMJPR1778), the KDDI Foundation, the Narishige Neuroscience Research Foundation, the Naito Foundation, and the Yazaki Memorial Foundation for Science and Technology to Yoichi Miyawaki, JSPS KAKENHI (JP16J00417) to Masashi Sato, and a contract research 173 with NICT to Masa-aki Sato. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Close investigation of human brain activity patterns plays a vital role in revealing how information is represented in the human brain. However, no method can directly measure fine spatial patterns of human brain activity at a high temporal resolution. Functional magnetic resonance imaging (fMRI) can capture the spatial patterns of human brain activity, but it lacks temporal resolution, as it measures metabolic processes reflected by blood-oxygenation-level-dependent signals. In contrast, magnetoencephalography (MEG) and electroencephalography (EEG) can capture the fast dynamics of brain activity patterns via electromagnetic signals that propagate from neurons; however, both of these techniques lack fine spatial resolution because their signals are measured by a small number of sensors placed around the head.
Given these constraints, researchers have tried to extract necessary information by solving an inverse problem to estimate cortical current source from measured MEG/EEG signals. MEG is more suitable than EEG for this purpose because of the homogeneous magnetic permeability of brain tissues. The typical approach for MEG source estimation uses an equivalent current dipole or distributed source model, assuming a few or many (typically 103–4) current dipoles in the brain. Although additional constraints are necessary to resolve the ill-posed condition, a distributed source model seems preferable to analyze fine spatial patterns of brain activity because of its descriptive power. However, no method can reconstruct cortical current source with perfect accuracy because of the ill-posed nature of MEG source estimation. It is hence important to be aware of limitations of MEG source estimation.
Previous studies have mainly evaluated the performance of MEG source estimation with regard to the accuracy of source position estimation. Thus, error has typically been quantified in terms of spatial displacement [1], dispersion [2–5], and overlap/non-overlap of the estimated sources relative to the original ones [6–8]. However, it remains unclear how accurately the spatial patterns of the original source are restored through MEG source estimation.
Restorability of the spatial patterns of the original sources through MEG source estimation is crucial for investigating the information content represented by brain activity patterns [9–17]. Previous fMRI studies have shown that multivariate pattern analysis (MVPA) allows information of images seen by participants to be extracted from the spatial patterns of their brain activity [18–20]. Recent MEG studies have also applied MVPA to estimated cortical current and evaluated the information represented by them [21–24]. This progress emphasizes the importance of evaluating whether the original spatial pattern of the source is preserved or whether a spurious pattern is fabricated through MEG source estimation.
In this study, we examine whether MEG source estimation can restore a pattern of the original cortical current and the represented information content of the source brain areas (Fig 1). For this purpose, we assumed patterns of source cortical current in certain areas on participants’ cortical surface models that would encode differences in artificial experimental conditions as multidimensional patterns. Given the source cortical current, we calculate the expected magnetic fields at the MEG sensors and estimate the spatial patterns of the cortical current from the simulated MEG sensor signals. We use four representative methods of MEG source estimation based on distributed source models. We quantify the similarity between the spatial patterns of the original and estimated cortical currents and show significant correlation between them. We then demonstrate that the estimated cortical current can be used to predict the corresponding artificial experimental conditions by pattern classification analysis but that significant prediction is possible even in cortical areas where the original source is not defined. This unexpected phenomenon makes it appear as if represented information spreads over a wide cortical area when MEG source estimation is used, which could lead to misinterpretations about the cortical areas that represent target information. Further analyses confirm that this information spreading can be similarly observed in real MEG data.
Two patterns of source cortical current are simulated (step 1) and then converted to MEG signals (step 2). Next, cortical current is estimated from the converted MEG signals using MEG source estimation methods (step 3). To evaluate pattern reproducibility and information restorability, the original and estimated patterns of cortical current are compared using multivariate pattern analyses (steps 4 and 5).
Methods
Modeling cortical surfaces
Magnetic resonance imaging (MRI) was used to obtain each participant’s cortical structural images. Cortical surface models were then extracted from the cortical structural images to simulate source cortical current. The simulation described in the following sections were conducted for each participant independently using the individual cortical surface models.
Participants.
Cortical structural images were obtained from nine participants (eight male and one female). They participated in our study voluntarily, and the same participants also took part in the MEG experiments described later. Each participant gave written informed consent before participating. The procedure was approved by the institutional review board of The University of Electro-Communications and Advanced Telecommunications Research Institute International (ATR) Brain Activity Imaging Center.
MRI acquisition.
Cortical structural images were obtained using 3.0-Tesla Siemens MAGNETOM Trio A Tim and Prisma fit scanners located at the ATR Brain Activity Imaging Center. T1-weighted magnetization-prepared rapid-acquisition gradient-echo (MP-RAGE) fine structural images of the whole head were acquired for each participant (208 sagittal slices; TR, 2250 ms; TE, 3.06 ms; TI, 900 ms; flip angle, 9°; field of view, 256 × 256 mm; voxel size, 1.0 × 1.0 × 1.0 mm; the same parameters were used on the Trio and Prisma scanners).
Extraction of cortical surfaces.
The cortical surface was defined as a polygon model of the gray matter surface extracted from each participant’s MRI data using the FreeSurfer software suite (http://surfer.nmr.mgh.harvard.edu/), with about 300,000 vertices. The polygonized cortical surface models were then imported into Brainstorm [25], and the number of vertices was downsampled to 15,002 to reduce computational load.
Simulation of source cortical current
A source cortical current was simulated for each participant’s cortical surface model with a time course designed to roughly imitate the evoked response caused by visual stimulation along the visual cortical hierarchy. The time period of the source cortical current was defined during −100–300 ms, which corresponds to a single trial in the simulation. We assumed two source areas in different time windows and ROIs: at 25–75 ms in the primary visual cortex (V1) and at 200–250 ms in the inferotemporal cortex (IT; Fig 2). ROIs were manually defined on the ICBM152 standard brain [26] and then projected onto each participant’s cortical surface model using FreeSurfer’s spherical morphing procedure (Fig 2A). In addition to V1 and IT, the parietal cortex (PR) was also defined without any sources for control analyses (Fig 2A). V1 was defined to encompass the occipital pole, IT was defined to overlap with the inferoposterior part of the temporal lobe, and PR was defined as the middle part of lateral parietal lobe. Each ROI enclosed 120 vertices (60 vertices for each hemisphere). No sources were assumed on other vertices. The time origin (0 ms) was considered to be the onset of visual stimulation. Although we used this particular timing setup, it does not affect the presented results qualitatively unless the two source activity temporally overlap.
(a) Three bilateral ROIs arranged on the cortical surface of the standard brain. (b) Source cortical current within each ROI. Line colors correspond to ROI colors shown in (a).
Each of the sources was a combination of multiple current dipoles with the same amplitude within each hemisphere, but different amplitudes across hemispheres. Current dipoles were placed at all vertices of the cortical surface whose directions were perpendicular to the cortical surface (the positive direction was defined as from the inside to the outside of the brain). Note that current dipoles with non-zero amplitude was placed only in the source ROIs whereas those placed in other areas had zero amplitude. The amplitude of the source cortical current in each ROI was temporally modulated as a sinusoidal waveform whose phase started at zero and ended at π within each time window (Fig 2B). Thus, the amplitude was maximal at the middle of each time window (50 ms and 225 ms for V1 and IT, respectively). We created two artificial experimental conditions with differences in maximum amplitude for each hemisphere: for condition 1, the maximum amplitude of the source cortical current was 1 nA⋅m for the left hemisphere and −1 nA⋅m for the right, while condition 2 was the opposite of condition 1 across hemispheres. Thus, the mean amplitude of the source cortical current across hemispheres was the same in both of the artificial experimental conditions.
Calculation of MEG sensor signals
To simulate MEG sensor signals generated by the original cortical current, we measured head position relative to the MEG sensors for each participant.
Head position measurement and registration.
We first measured each participant’s head shape and the positions of five electromagnetic marker coils attached to the participant’s head (three on the forehead and two on the ear tragi) using a three-dimensional digitizer (FastSCAN, Polhemus Inc., USA). The magnetic field generated by the marker coils was then measured by an MEG system (PQ1400RM, Yokogawa Electric Co., Japan) with 400 SQUID sensors (210 axial and 190 planar gradiometers), and each participant’s head position was estimated from the measured signals. Each participant’s head position in the MEG system was then coregistered to the cortical structural image of the same individual’s MRI data. The positional relationship between the MEG sensors and the cortical surface was determined from each participant’s coregistration results.
MEG signal synthesis.
MEG sensor signals were calculated by a forward model that describes magnetic propagation processes from the original cortical current to the MEG sensors. The forward model can be described as
(1)
where b (ℝN×1 vector; N, number of MEG sensors) represents measured MEG signals, G (ℝN×M matrix; M, number of vertices on the cortical surface) represents a leadfield matrix, j (ℝM×1 vector) represents the original cortical current, and e (ℝN×1 vector) represents sensor noise. Each column of the leadfield matrix indicates the amplitude of the MEG signals generated by a unit current dipole of the corresponding vertex. G was calculated by the boundary element method using the OpenMEEG software package [27,28]. Sensor noise was modeled to follow a Gaussian distribution N(0,σ2). In this simulation, σ was set to twice the largest MEG signal amplitude across all time points and sensors to represent the low signal-to-noise ratio that would occur in an actual experiment. The noise was added to each time point and MEG sensor independently. We constructed five artificial experimental runs by repeating this procedure 50 times for each artificial experimental condition. Each run consisted of 100 trials in total.
Estimation of cortical current from simulated MEG sensor signals
Suppose that is an estimate value of j (ℝM×1 vector) given b and G. To estimate
, we need to solve an inverse problem that is ill-posed because there are fewer MEG sensors than assumed current dipoles on the cortical surface. To obtain a unique solution, we used the following four representative frameworks for MEG source estimation, which have been used extensively in recent studies.
L2-norm regularization.
We first tested L2-norm regularization, whose objective function is written as
(2)
where C is the ℝN×N noise covariance matrix, R is the ℝM×M source covariance matrix, λ is a regularization parameter, and T indicates the transpose. The first term on the right-hand side of Eq (2) indicates the estimation error weighted by noise covariance, and the second term indicates L2-norm regularization weighted by source covariance. This framework is known as minimum norm estimation (MNE) [29]. Although classical studies set C and R as identity matrices, here we used a modified version of MNE, in which C was computed from simulated MEG signals with a period of −100 to −1 ms, R was weighted by depth from the cortical surface, and the leadfield matrix was spatially whitened. We tested different λ values ranging from 10−2 to 103 in exponential steps. We used MNE implemented in Brainstorm (http://neuroimage.usc.edu/brainstorm/).
L1-norm regularization.
L2-norm regularization yields a cortical current broadly distributed over the cortical surface. Such a solution is often considered undesirable, because one of the purposes of MEG source estimation is to identify the brain areas related to the experimental conditions under investigation. As an MEG source estimation method that gives preference to more sparse cortical currents than MNE, we tested L1-norm regularization, also known as minimum current estimation (MCE; [30,31]) or least absolute shrinkage and selection operator (LASSO; [32]), whose objective function is written as
(3)
where λ is a regularization parameter, and |x|p represents the Lp-norm of a vector X. L1-norm regularization promotes sparse solutions in which only small numbers of elements of
have non-zero values. In this formulation, λ should be smaller than
[33,34]; otherwise
becomes the zero vector. As λmax was on the order of 10−11 in our simulation, we tested λ values smaller than the λmax, ranging from 10−16 to 10−11 in exponential steps. We used the L1-norm minimization implemented in the scikit-learn software package (http://scikit-learn.org/stable/).
Hierarchical Bayesian estimation.
The above two methods add a constraint for the minimization of the L2 and L1-norm of , respectively. The constraint can be further improved if prior information on the locations of the MEG signal sources is available. Previous studies have shown that fMRI activity evoked by experimental conditions similar to those used to induce MEG signals can serve as prior information about the possible locations of MEG signal sources [35,36].
In this study, we used a hierarchical Bayesian model to introduce prior information on the MEG signal sources [36]. In this model, the prior information for is introduced as a prior probability that follows a zero-mean Gaussian distribution with inverse variance r:
(4)
where R = diag(r) and r is a ℝM×1 vector whose elements
are hyperparameters denoting source variance. The inverse variance r is also treated as a random variable that follows a gamma distribution
(5)
where
represents a gamma distribution with mean
and degrees of freedom γ0,i. The γ0,i is also known as a confidence parameter. The prior distribution combined with this hyperprior distribution is equivalent to automatic relevance determination (ARD) prior [36–38]. Information about the fMRI activity is introduced to control
, with a large source variance being more likely where larger fMRI activity is observed. The likelihood function is defined with the assumption that the MEG sensor noise follows a Gaussian distribution as
(6)
where α indicates noise variance. Using Bayes’ theorem, the posterior probability distribution for the estimated cortical current can be described as
(7)
where P(b) indicates the marginal likelihood defined as
(8)
Using
, the posterior distribution of
can be obtained as
(9)
The cortical current is estimated by taking the expectation of the posterior distribution.
We conducted this hierarchical Bayesian estimation using variational Bayesian multimodal encephalography software (VBMEG; http://vbmeg.atr.jp/). For prior information, we set the magnitude of the fMRI activity to 1 for all vertices in V1 and IT and to 0 for all other vertices. Note that cortical current was estimated for all vertices on the whole cortical surface regardless of the prior information. The width of the time window for estimating source variance were set to 100 ms and the time window was shifted in 50-ms steps. We used a single value of the confidence parameter for all vertices (i.e., γ0,i = γ0). Tested values were γ0 = 0 and that in the range from 10−1 to 103 in exponential steps.
LCMV beamformer.
The fourth representative method for MEG source estimation that we tested was the linearly constrained minimum variance beamformer (LCMV; [39]), which is based on a different concept from the regularization methods described above. LCMV constructs a spatial filter wi for b to estimate the ith cortical current under the constraint
, where gi is the column vector of the leadfield matrix corresponding to the ith vertex. Under this constraint, LCMV minimizes the power of
. This minimization effectively suppresses the influence of spurious sources and sensor noise because the lower bound of the power of
should match the original cortical current as a result of the constraint. This procedure yields a spatial filter wi:
(10)
where B indicates the covariance matrix of b. Because the signal-to-noise ratio of b can be low in an actual experiment, noise can significantly affect wi. Thus, wi is normalized by noise variance as
(11)
Thus,
can be obtained as
(12)
can thus be estimated by calculating
for all vertices. In this study, B was computed using a 100-ms time window shifted with 50-ms steps. Noise covariance matrix C was computed from simulated MEG signals with a period of −100 to −1 ms. We performed the above procedure using custom written MATLAB (MathWorks, Natick, MA) programs.
In this study, MEG source estimation was conducted using all trials for each artificial experimental run independently to avoid using the same data twice in the training and test (double dipping) in leave-one-run-out cross-validation (see Time-resolved decoding in each ROI section).
Evaluation of source estimation accuracy
To evaluate source estimation accuracy, we used the two indices of area under the precision-recall curve and correlation of spatial patterns. Note that because the value of the MEG sensor noise was set high to resemble actual measurements, these evaluations would have been strongly influenced by noise if based on the data from a single trial. We therefore averaged the estimated cortical current across all trials for each artificial experimental condition and used the averaged data for evaluations.
Area under the precision-recall curve.
First, we used the area under the precision-recall curve (APR) to evaluate the accuracy of localization of source cortical current by the MEG source estimation methods. As we simulated the source cortical current as a distribution rather than a current dipole, it is suitable to evaluate the degree of overlap between the source ROIs and the source positions defined by the estimated cortical current. Representative methods for such an evaluation are the area under the receiver operating characteristic curve (AUC) [40] and APR [41,42]. A previous study showed that AUC is less sensitive than APR when the number of positive samples is significantly different from the number of negative ones [43]. As the numbers of vertices within (positive samples) and outside (negative samples) a source ROI differed substantially (120 vs. 14,882) in our simulation, we used APR instead of AUC. The detailed procedure to calculate APR is shown in Appendix A. The chance level of APR was calculated as 0.008. APR was calculated at the time when the source cortical current was at its maximum for each ROI (50 ms and 225 ms for V1 and IT, respectively) in each artificial experimental condition.
Correlation of spatial patterns.
Second, we calculated the correlation coefficients between the original and estimated cortical currents to evaluate how accurately spatial patterns were restored by MEG source estimation. Spatial correlations were computed between the original and estimated cortical current patterns over vertices within each source ROI for each artificial experimental condition at 50 ms and 225 ms for V1 and IT respectively. We assumed that the spatial patterns of the original cortical current were restored if a significant correlation (P < 0.05, uncorrected) was observed.
Time-resolved decoding in each ROI
We also used pattern classification analysis to evaluate whether the MEG source estimation methods restored information represented by differences in the spatial patterns of the original cortical currents. Pattern classification was performed to predict the artificial experimental conditions from the estimated cortical current at each time interval in each ROI on a trial-by-trial basis. We used the linear support vector machine (SVM; [44]) implemented in libsvm (http://www.csie.ntu.edu.tw/~cjlin/libsvm/) as a pattern classifier or “decoder”. The decoder’s prediction accuracy was evaluated by time-resolved leave-one-run-out cross-validation analysis [16,45], in which the decoder was trained with four out of five artificial experimental runs and tested with the remaining one at a particular time interval of 5 ms (see below about preprocessing of the estimated cortical current). This procedure was repeated until all artificial experimental runs were tested once at each time interval.
The estimated cortical current was preprocessed before decoding as follows. First, the time course of estimated cortical current was downsampled at each 5-ms interval with an equally-weighted smoothing window of 10 ms. Then, the downsampled data underwent z-score normalization across trials at each time interval for each vertex. The parameters for normalization (i.e., mean and standard deviation) were calculated using only a training data set and were then applied to both the training and test data sets. A decoder received the preprocessed data corresponding to each ROI as an input feature (ℝ120×1 vector, for all ROIs). Custom software programs partially based on the Brain Decoder Toolbox (http://www.cns.atr.jp/dni/download/brain-decoder-toolbox/) were written to perform the analysis.
Searchlight decoding
MEG source estimation cannot reconstruct the original cortical current with perfect accuracy, and spurious cortical current may be found beyond the source ROIs. If this spurious cortical current has systematic differences related to the artificial experimental conditions, a decoder can capture these differences and predict the artificial experimental conditions, even from irrelevant brain areas far from the source ROIs.
To test this possibility, we performed searchlight decoding [46] to examine whether and how far such systematic differences in estimated cortical current spread beyond the source ROIs. The estimated cortical current at each vertex and its 119 neighborhood vertices were used as an input feature (ℝ120×1 vector, the same dimensions as for the ROI-based decoding). Decoding was performed for all vertices using the same procedure as that employed for the ROI-based decoding, but the time points only of 50 ms and 225 ms were tested in this analysis.
To assess how far the systematic differences had spread, we analyzed the relationship between the distance from the source ROI to a particular vertex and the prediction accuracy at that vertex. Distance was calculated from the center of mass of the source ROI. This calculation was performed separately for each hemisphere.
Statistical significance of decoding analysis
We defined the statistical significance level of these decoding analyses (ROI-based and searchlight) with reference to the prediction accuracy obtained from the whole cortical area at 0 ms using searchlight decoding for each participant. As the MEG signals at 0 ms consisted of noise, prediction accuracy at this time served as a null distribution. We used the 99th percentile of the null distribution as the statistical significance level. We also used binomial distribution and permutation tests to define the significance level and examined whether the significance of decoding accuracy depends on the definition of the significance level. The significance levels of the permutation tests were defined as 99th percentile of 200 random permutations at each time point.
Real data analysis
We also applied pattern classification analysis to real data. In this analysis, we used VBMEG as a source estimation method because it showed the most accurate source localization and pattern reconstruction in the simulations (see Results).
Participants.
Four male participated in this experiment voluntarily. Each participant gave written informed consent before participating in the experiment, which was approved by the institutional review board of The University of Electro-Communications and ATR Brain Activity Imaging Center.
Visual stimuli.
Pairs of wedges with white and black checkerboard patterns were used as stimuli. The center angle of the wedges was 30°, and the wedges rotated clockwise in 30° steps (S11(A) Fig). The stimuli were projected onto a translucent screen, subtending 18° × 18° of visual angle, with a gray background. Each trial consisted of the fixation period and the stimulus presentation period. In the fixation period, a fixation point consisting of black and white concentric circles was presented for 1,500 ms at the center of the screen. The white part of the fixation point changed to red 700–1,200 ms before the stimulus presentation period began. Participants were instructed to suppress eye blinks while the color of the fixation point was red. After the fixation period, the stimulus presentation period began and each stimulus was presented for 300 ms and was followed by a 300 ms inter-stimulus interval. The red part of the fixation point changed back to white when the stimulus presentation period ended. One run consisted of 20 trials, and a total of 12 runs were conducted for each participant.
MEG acquisition and preprocessing.
We acquired MEG signals during the experiment using the same MEG system used to estimate head position for the simulation. MEG signals were acquired at 1000 Hz and a band-pass filter with 0.05 to 200 Hz was applied online. Electrooculography (EOG) was simultaneously obtained to allow rejection of trials corrupted by eye blinks and eye movement. Each participant’s head position in the MEG system was acquired before each run. We applied time-shifted principal component analysis [47] and a low-pass filter with a 100 Hz cutoff frequency to the measured MEG signals to reduce environmental and high-frequency noise. MEG signals corresponding to each stimulus presentation were epoched from −100 to 400 ms (0 ms was defined as stimulus onset) to constitute a trial. Channels whose MEG signal amplitude exceeded the range between −1 and 1 pT in more than 5% of all trials were discarded, as they were considered to be contaminated by noise. Trials were discarded from the following analysis if the EOG fell outside the range between −1 and 1 mV, or at least one MEG channel fell outside the range between −1 and 1 pT, as they were considered to be contaminated by eye blinks, eye movement, or noise.
Source estimation.
Forward model
We used a cortical structural image of each participant’s whole head acquired using the Trio and Prisma scanners at the ATR Brain Activity Imaging Center with the acquisition parameters mentioned in simulations. The cortical surface of each participant’s structural image was polygonized with 15,002 vertices using FreeSurfer and VBMEG software such that the number of vertices matched to that used in the simulations. Head positions were acquired in the MEG system for each run and they were coregistered to the cortical surface. A leadfield matrix was then computed for each participant and each run using the boundary element method.
Source estimation
As prior information, we set the magnitude of fMRI activity to 1 for all vertices in the primary visual cortex (V1) and the higher visual cortex (HVC), leaving it as 0 for all other vertices. We chose these two ROIs to investigate whether spurious cortical current produced by MEG source estimation has systematic differences related to the experimental condition (see the section below). HVC was defined as a combination of the lateral occipital complex (LOC), fusiform face gyrus (FFA), and parahippocampal place area (PPA). V1, LOC, FFA, and PPA were identified for each participant using standard experimental procedures used in previous studies [48–51]. The width of the time window for estimating source variance was set to 10 ms, and the time window was shifted in 5-ms steps. We used γ0,i = 102 for all vertices because it showed the most accurate correlation between the estimated and original cortical current in the simulation. Source estimation was conducted using VBMEG for each participant and each run.
Time-resolved decoding in each ROI.
To investigate whether the spurious cortical current contained systematic differences related to the experimental conditions in the real data, we conducted time-resolved decoding in each ROI. Pattern classification was performed to predict the experimental conditions from the estimated cortical current at each time point in V1 and HVC. Three pairs of wedges presented in the upper right and the lower left areas were labeled as condition 1, and the other three pairs of wedges presented in the upper left and the lower right areas were labeled as condition 2 (S1B Fig). As the latency of neural activity in V1 is expected to be shorter than that in HVC [52], the experimental condition is expected to be decoded from the estimated cortical current in V1 earlier than it can be decoded from that in HVC. Thus, if significant prediction accuracy is obtained in HVC at the same latency as in V1, we can assume that the spurious cortical current estimated in HVC has information about the experimental condition. In this analysis, we used a SVM as a classifier and its prediction accuracy was evaluated by leave-one-run-out cross-validation for each time point. Before decoding, the mean amplitude of the estimated cortical current during the baseline period, in which the stimulus-evoked neural activity did not appear in V1 (−100 to 40 ms), was subtracted from each trial and each vertex. The end of the baseline period was determined according to previous studies [52,53]. Z-score normalization was also applied, as in the simulation. As in the simulation, we defined the statistical significance level as the 99th percentile of prediction accuracy obtained from searchlight decoding at 0 ms (see Searchlight decoding section described below). We defined the onset latency of significant prediction accuracy in each ROI as the earliest time point from which the classifier showed significant prediction accuracy on at least 40 consecutive points (40 ms).
Time lag between changes in prediction accuracy and estimated cortical current.
The time course of the estimated cortical current was also compared with that for the prediction accuracy of each ROI. To quantify the time course of the magnitude of the estimated cortical current for each ROI, we used F-statistics calculated by comparing the variance of the estimated cortical current across all vertices within the ROI at each time point with that during the baseline period. Before calculating the F-statistics, the estimated cortical current underwent subtraction of the mean amplitude of the baseline period followed by trial averaging for each vertex. Next, we calculated the cross-correlation between the time courses of the F-statistics and prediction accuracy. In this calculation, all the time points after the baseline period were used, and the lag in the maximum of the cross-correlation was acquired. Positive lag means that the time course of the prediction accuracy precedes that of the estimated cortical current.
Searchlight decoding
We performed searchlight decoding to examine whether and how far systematic differences about experimental conditions spread in the estimated cortical current also for the real data. The estimated cortical current at each vertex and its 119 neighborhood vertices were used as an input feature (ℝ120×1 vector, the same dimensions as the simulation). Decoding was performed for all vertices using the same procedure as for the ROI-based decoding at the onset latency of prediction accuracy for time-resolved decoding in V1 for each participant because it is highly likely that V1 could be a major source that represents information corresponding to visual stimuli but few areas are active at that time. As in the simulation, we defined the significance level as the 99th percentile of prediction accuracy of searchlight decoding at 0 ms obtained from the whole cortical area for each participant. This significance level was also used for time-resolved decoding in each ROI.
Results
Simulation data analysis
Localization accuracy of source ROIs.
First, we evaluated how accurately the MEG source estimation methods localized source cortical current using APR (Fig 3A; the results for V1 at 50 ms and IT at 225 ms in condition 1 are shown. The results of condition 2 are shown in S2 Fig). MNE, MCE, and VBMEG had at least one hyperparameter whose mean APR exceeded its baseline value in both ROIs. The mean APR of LCMV also exceeded its baseline value in both ROIs. VBMEG showed the highest APR, whereas MCE showed the lowest. We further illustrated the time course of the estimated cortical current within source ROIs for a single participant (Fig 3B) and the estimated cortical current maps by rendering them on the cortical surface of the same participant (Fig 3C). The time course of the estimated cortical current showed that the major sources were successfully estimated in the ROIs at the correct times, although they were missed in some cases (e.g., sources in IT at 200–250 ms by MCE) and small amplitudes were also found at the incorrect times (e.g., V1 at 200–250 ms and IT at 25–75 ms for LCMV). According to the results of the estimated cortical current maps, MNE, MCE, and VBMEG at 50 ms typically showed larger localized amplitudes around V1 than in other areas, whereas LCMV tended to produce a broad estimation (Fig 3C). At 225 ms, VBMEG showed better localization of a large amplitude around IT than other methods (Fig 3C). These results demonstrate that MEG source estimation can localize the major cortical current in the correct ROIs at the correct times, although some exceptions were found.
(a) Area under the precision-recall (APR) curves for V1 at 50 ms and IT at 225 ms in artificial experimental condition 1 (averaged across participants; error bars, s.d.; gray dashed lines, baseline value of APR). 0* indicates γ0 = 0. Results for artificial experimental condition 2 are shown in S2 Fig. (b) Time course of the mean estimated cortical current within V1 and IT with the hyperparameters that achieved the highest APR. The results for artificial experimental condition 1 for a single participant are shown as examples. (c) Absolute values of estimated cortical current with amplitude normalized to 0–1 for visibility. Vertices showing zero amplitude are set as transparent, and sulci (dark gray) and gyri (light gray) are visible on the maps. Results are shown for the same hyperparameters and participant as in (b).
Spatial correlation between original and estimated cortical currents.
We then evaluated the accuracy of restoration of the spatial patterns in the original cortical current. Correlation coefficients were calculated for spatial patterns between the original and estimated cortical currents within the source ROIs (Fig 4; only results for V1 and IT in condition 1 are shown, with the rest shown in S3 Fig). MNE showed the highest mean correlation at around λ = 10−1 to 100 in both ROIs. MCE showed the highest correlation in V1 at λ = 10−12, but the mean correlations for IT did not exceed significance level. VBMEG generally showed higher correlations than the other source estimation methods and achieved the highest value around γ0 = 102 in both ROIs. LCMV showed correlations comparable to those of MNE. These results indicate that all of the investigated MEG source estimation methods, except for MCE, can restore spatial patterns of the source cortical current that significantly correlate with the original ones.
Results for V1 at 50 ms and IT at 225 ms in artificial experimental condition 1 are shown (averaged across participants; error bars, s.d.; gray dashed lines, significance level [uncorrected P < 0.05]). 0* indicates γ0 = 0. Results for artificial experimental condition 2 are shown in S3 Fig.
Time-resolved decoding in each ROI.
We further used pattern classification analysis to examine whether the MEG source estimation methods preserved the information represented by differences in the spatial patterns of the original cortical current. Because the difference was exclusively represented in V1 at 25–75 ms and in IT at 200–250 ms, only those particular combinations of ROI and time window were expected to show significant performance in prediction of the artificial experimental conditions (for ideal results, see Fig 5A).
(a) Ideal time courses of prediction accuracy. (b–e) Time courses of prediction accuracy for MNE, MCE, VBMEG, and LCMV, respectively. Results for the hyperparameters that achieved the highest correlation coefficient in V1 (see Fig 4) are shown. Solid lines indicate the mean prediction accuracy across participants. Shading indicates the 1st–99th percentiles of the prediction accuracy across participants. Red dashed lines indicate the mean significance level across participants. Results for other hyperparameters are shown in S4 Fig.
The results showed that all MEG source estimation methods achieved significant prediction for these combinations of ROI and time window (in Fig 5B–5E, only results for the hyperparameter that achieved the highest correlation in V1 are shown [see Fig 4]; the full results are shown in S4 Fig). Thus, the represented information was preserved through the MEG source estimation.
However, significant prediction was also possible from the ROIs at non-informative time windows (i.e., V1 at 200–250 ms and IT at 25–75 ms), during which no differences were defined in the spatial patterns of the source cortical current, particularly for MNE, VBMEG, and LCMV. Furthermore, even in PR, where no source cortical current was provided at any time, significant prediction was possible for both time windows. MCE showed only a small increase in prediction accuracy for the non-informative combinations of ROI and time window, but it also achieved less accurate prediction than the other MEG source estimation methods with the informative combinations. We also performed the same analysis with addition of Gaussian noise N (0,1) to PR as source cortical current and examined whether such irrelevant activity influences decoding. The results showed that prediction accuracy was decreased but still remained above significance even in the case of non-informative combinations of ROI and time window (S5 Fig). These results demonstrate that information about artificial experimental conditions originally contained within a particular brain area can spatially spread to other brain areas through MEG source estimation. We call this phenomenon “information spreading.” Preliminary analyses also confirmed that these results were qualitatively similar if low-noise MEG signals or more complicated spatial patterns of source cortical current were used.
There was no qualitative difference between the significance levels calculated from the searchlight decoding at 0 ms (55.6±1.0%, presented in the figures), a binomial distribution (54.4%), and the permutation tests (56.1±0.8%, mean±s.d. across all time points in the trial). We also confirmed that no significant difference was observed in the number of significant time points across the three significance levels (P < 0.05, Kruskal-Wallis test) (S6 Fig). Because the statistical difference was not found to depend on the tested significance levels, we adopted the significance level calculated from the searchlight decoding at 0 ms in this paper.
Searchlight decoding and the spatial extent of information spreading.
We conducted searchlight decoding to investigate the spatial extent of information spreading produced by MEG source estimation. Fig 6A shows maps of prediction accuracy for a single participant (the same participant shown in Fig 3B and 3C). Broad cortical areas showed near-perfect prediction accuracy for all MEG source estimation methods except MCE, whose prediction accuracy was much lower than that of the other methods, even within source ROIs (Fig 5). We quantified the spatial profile of prediction accuracy on the cortical surface (Fig 6B; results for the left hemisphere are shown. Full results are shown in S7 Fig). High prediction accuracy was observed in cortical areas distant from the source ROIs, particularly MNE, VBMEG, and LCMV, indicating that information spreading extended over wide cortical areas. MCE showed the narrowest information spreading, but its prediction accuracy was also the lowest. These results indicate that information spreading can potentially lead to false-positive identification of brain areas representing target information content, as significant decoding performance was found in irrelevant brain areas.
Results for the hyperparameters that achieved the highest correlation coefficients (see Fig 4) are shown. (a) Maps of prediction accuracy at 50 ms and 225 ms. Results of a single participant are shown as examples. Colored lines delineate ROIs. (b) Spatial extent of information spreading at 50 ms (left) and 225 ms (right). The horizontal axes represent distance from the center of mass of V1 left (left) and IT left (right). Solid lines indicate mean prediction accuracy across participants. For an illustration purpose, prediction accuracy was averaged for each 5-mm distance bin. Shading indicates the 1st–99th percentiles of prediction accuracy across participants. Each dashed line indicates a significance level averaged across participants for each MEG source estimation method (they mostly overlap). Results for all hyperparameters in both hemispheres are shown in S7 Fig.
We further examined whether such wide information spreading was observed only in multivariate analysis or could also occur with univariate analysis. First, we calculated t-values for difference in amplitude of the estimated cortical current between artificial experimental conditions. Significant t-values were observed in cortical areas distant from the source ROIs (S8 Fig). Next, we conducted univariate searchlight decoding by using the mean value of estimated cortical current within each searchlight consisting of a target vertex and its 119 neighborhood vertices as an input feature. High prediction accuracy was observed in cortical areas distant from the source ROIs, but the prediction accuracy was lower than that of searchlight decoding using multivariate patterns as input features (S7 Fig and S9 Fig). For example, prediction accuracy for multivariate searchlight decoding stayed at 100% within at least 0.04 m from the source ROI for VBMEG irrespective of the hyperparameters whereas that for univariate searchlight decoding dropped rapidly as the distance from the source ROI. Other cases were also similar. These results indicate that information spreading can be observed if MEG source estimation is combined with univariate analysis (t-test and univariate decoding), but its effect is larger if combined with multivariate pattern analysis.
Real data analysis
To investigate whether information spreading also occurs in real data, we conducted pattern classification analysis of the estimated cortical current in each ROI to predict the visual stimuli presented to participants. Both ROIs showed a similar time course for prediction accuracy (Fig 7A), with almost the same onset latency for significant prediction accuracy between the ROIs (Fig 7B). These results are incompatible with electrophysiological knowledge suggesting that the latency of neural activity in HVC is longer than in V1 [52,53]. In contrast to the time course of prediction accuracy, the time course of the F-statistics in HVC was delayed from that in V1 (Fig 7C). Accordingly, the lag between prediction accuracy and F-statistics was smaller in V1 than in HVC, and prediction accuracy tended to precede F-statistics in HVC (Fig 7D). Thus, significant prediction accuracy was obtained before HVC was activated. In addition, although searchlight decoding was performed at a very short latency, at which V1 could be the truly informative ROI, significant prediction accuracy was obtained in other brain areas outside of V1 (Fig 7E). These results suggest that the significant prediction accuracy in HVC at this short latency was caused by spurious cortical current that might be originated from V1, indicating that information spreading also occurs in real data.
(a) Time course of prediction accuracy. Solid lines indicate prediction accuracy for V1 (red) and HVC (blue), respectively. Gray dashed line indicate significance level. Results of a single participant (participant 1) are shown as examples. (b) Mean onset latency of significant prediction accuracy for each ROI (error bars, s.d. across participants). (c) Time course of F-statistics. F-statistics are normalized to 0–1 for visibility. Vertical dashed lines indicate the onset of prediction accuracy for each ROI (red, V1; blue, HVC). Results of a single participant (participant 1) are shown as examples. (d) Mean lag of cross-correlation between the time course of prediction accuracy and F-statistics (error bars, s.d. across participants). Positive values indicate that prediction accuracy precedes the F-statistics. (e) Searchlight decoding. Cortical areas that showed significant prediction accuracy are colored. White and black enclosed areas are V1 and HVC, respectively. Results of a single participant (participant 1) are shown as an example. Results of other participants for (a), (c), and (e) are shown in S10A, S10B and S10C Fig, respectively.
Discussion
In this study, we evaluated whether MEG source estimation can restore information represented in original cortical current patterns. We found information spreading, which allowed decoders to achieve high prediction accuracy in irrelevant brain areas where no source was originally defined (Fig 5), even with accurate source localization (Figs 3 and 4). Searchlight decoding analysis further revealed that the effects of information spreading extended to widespread brain areas (Fig 6), especially for multivariate pattern analysis (S7 Fig and S9 Fig). In real data, significant prediction accuracy was obtained from V1 and HVC at the same latency, and other brain areas also showed significant prediction accuracy at a very short latency after the onset of the visual stimulation (Fig 7). These results indicate that MEG source estimation can restore information represented in patterns of original cortical current, but that this information spreads from informative brain areas to non-informative brain areas, demonstrating the possibility of false-positive interpretations when identifying brain areas that represent target information content based on the results of the decoding of estimated cortical current.
Previous studies have typically evaluated MEG source estimation methods in terms of the localization error and spatial dispersion of estimated cortical current (e.g. [2,4,5,7]). As the main focus of their source estimation was to minimize localization error and spatial dispersion, they did not pay much attention to the patterns of estimated cortical current within particular brain areas. This study has extended the evaluation of MEG source estimation methods and is the first to evaluate patterns of estimated cortical current. Our analyses suggest that the representative MEG source estimation methods tested here are imperfect with respect to their accuracy of localization of represented information content, even if they appear to provide highly accurate estimations of major activity in expected brain areas.
The origin of such imperfect localization of represented information (i.e., the cause for information spreading) probably lies in the underdetermined nature of MEG source estimation. MEG source estimation using a linear inverse method estimates the cortical current from the observed magnetic field b using a linear inverse filter L as
(13)
and here, b can be rewritten using a noiseless linear observation model as
(14)
where A (ℝM×M matrix) is called a resolution matrix [2]. A should be the identity matrix for perfect source reconstruction, but such reconstruction is impossible because the influence of different cortical sources on MEG channels can be very similar when the sources are located close to one another. Such a situation may be possible for all cortical locations, leading to non-zero off-diagonal elements of A. Accordingly, the original cortical current j is spatially spread by application of A, and spurious activity is estimated in irrelevant cortical areas. If such spurious activity shows a systematic difference between experimental conditions, even though its amplitude is very small, it can be informative enough for a pattern classifier to predict the corresponding experimental conditions because pattern classification analysis is more sensitive to slight differences in brain activity than conventional univariate analysis [20], resulting in information spreading (Figs 3B, 6 and 7).
In our simulation, we tested representative MEG source estimation methods based on L2- and L1-norm minimization to suppress such spurious cortical current. Both methods effectively diminished spurious cortical current in terms of its amplitude (Fig 3) but were insufficient to suppress information spreading (Fig 6). We also tested a model based on a hierarchical Bayesian framework using ARD prior. Even though its source localization accuracy was the highest among the methods tested (Fig 3A), the information spreading was qualitatively similar to that produced by the L2- and L1-norm minimization methods (Fig 6). Our results also indicate that the LCMV, another representative method for MEG source estimation, failed to diminish information spreading (Fig 6). Thus, none of the tested MEG source estimation methods provided a critical solution to suppress information spreading. Small but systematic differences in estimated cortical current patterns can yield confusing decoding performances.
Although we tested representative methods of MEG source estimation, other untested methods could possibly achieve accurate pattern estimation of original cortical current while suppressing information spreading. For example, a model combining L2- and L1-norm minimization, known as the elastic net [54], could work better than either model used individually because L1 and L2 often prefer solutions that are sparse and smooth, respectively (Fig 3B). A combination of these two methods could balance out these characteristics. VBMEG can also be updated by explicitly introducing similarity and group structures between parameters as their covariance. Similarity and group structure information could be inferred from actual neuroscientific knowledge about cortical parcellation [55] and structural/functional connectivity [8]. However, these updates increase the number of hyperparameters, and such model complications would present further difficulties in hyperparameter tuning.
As a potential method to reduce effects of information spreading, we attempted a signal leakage reduction based on orthogonalization, which has been proposed as a useful method to diminish artifactual zero-lag correlation in functional connectivity (Appendix B). The method reduced the correlation between estimated cortical current in two different ROIs but also decreased prediction accuracy even in an ROI containing informative source cortical current. Therefore, signal leakage reduction could be insufficient to identify truly informative ROIs while reducing information spreading.
However, this fact does not deny a possibility that signal leakage reduction based on another signal processing methods works efficiently to mitigate information spreading. In fact, a recent study showed that a method based on independent component analysis (ICA) can reduce signal leakage through MEG source estimation while preserving significant decoding performance to predict categories of presented visual stimuli [56]. Although the study evaluated effects of signal leakage reduction only in nearby ROIs, the results suggest that the ICA-based method might possess a potential to reduce signal leakage and information spreading over wide brain areas.
Although we tested only classification analysis in this study, it would be interesting to examine whether similar phenomena are observed for other MVPA methods. For example, representational similarity analysis (RSA; [57]) has been one of the most popular MVPA methods used in fMRI studies (e.g. [58,59]), and recent studies have successfully applied it to MEG signal analysis [60,61]. Those studies applied RSA to measured MEG signals in the sensor space, but it seems worthwhile to examine whether the same analysis can be performed for estimated cortical current because the relationship between brain areas and information representation can be discussed more straightforwardly. To this end, it is necessary to reveal whether the similarities between the original cortical current obtained in experimental conditions (i.e., the representational geometry) hold after application of source estimation. It is also important to clarify whether the representational geometry propagates between brain areas as information spreading. Preliminary analyses have been conducted in our recent study [62]. These analyses will further clarify the limitations and characteristics of MEG source estimation from a multivariate perspective, helping us to extract and closely investigate the information content represented in the spatiotemporal dynamics of human brain activity.
In summary, this study showed that spurious cortical current is inevitably generated through MEG source estimation and that even though its amplitude is negligibly small, it contains systematic differences related to experimental conditions. These differences can be easily detected by a pattern classifier, and the unexpected prediction accuracy may lead to false-positive interpretations of brain areas that represent information on the experimental conditions. Therefore, our results may assist to define the reliability of scientific findings based on MEG source estimation and multivariate pattern analysis.
Appendix A. Calculation of APR
To calculate APR, we first converted to an absolute normalized value
as
(A1)
We shifted the threshold β in the interval [0 1], and if the ith vertex showed
, we judged the ith vertex as one of the source positions in
. In this analysis, we defined a set of vertices of source ROIs as Θ (i.e., vertices of V1 or IT) and defined a set of other vertices as
(i.e., vertices other than V1 or IT). The ith vertex was judged as
We defined a set of TP vertices at β as TP(β) and the same for FP as FP(β). A precision–recall curve can be created by plotting recall (REC(β)) against precision (PREC(β)). REC(β) and PREC(β) were computed as
(A2)
(A3)
where size( ) indicates the number of elements in the set. The baseline value of APR [43] can be defined as
(A4)
Appendix B. Testing signal leakage reduction
When functional connectivity is analyzed using MEG signals, it is known that signal leakage of estimated cortical current by MEG source estimation causes zero-lag correlation between cortical areas. This is an unexpected artifact because neural signal conduction should yield non-zero time lag between functionally connected areas (except areas sharing common inputs). To solve this problem, several studies have proposed methods for signal leakage reduction [63–65]. Here, we investigated whether a similar method can also help to reduce information spreading.
The basic idea common to the signal leakage reduction methods [63–65] is orthogonalization of a time series of estimated cortical current in one source location relative to another. In the case of orthogonalization of ROIs containing multiple sources, principal component analysis (PCA) was applied to multiple time series of estimated cortical current in each ROI, and the orthogonalization was applied to one-dimensional time series constructed with coefficients of principal components corresponding to the majority of the variance of the estimated cortical current [65]. Thus, previous studies performed the orthogonalization in a one-to-one, univariate manner. However, such orthogonalization cannot be applied straightforwardly to our case because multivariate pattern analysis needs to account for distributed patterns of estimated cortical current.
To deal with this condition, we tested sequential orthogonalization, which was an extension of the previous orthogonalization method proposed in [63]. Suppose that we have ROI X and ROI Y, which consist of M and N vertices, respectively. Our objective is to orthogonalize the estimated cortical current in ROI Y relative to that in ROI X. Let xm and yn be time series of the mth vertex in ROI X and the nth vertex in ROI Y, respectively. xm and yn have the same length (ℝT×1 vector, where T is length of the time series). The sequential orthogonalization was conducted as follows:
(B1)
where xm+ indicates the pseudo-inverse of xm.
If components parallel to each of x1…xM are subtracted simultaneously as yn,M = yn−x1x1+yn−x2x2+yn⋯–xMxM+yn, components correlated between x1…xM are subtracted from yn redundantly because x1…xM are not orthogonal to each other. Sequential orthogonalization can circumvent this problem because, at step m, a component parallel to xm−1 has already been subtracted one step before, and thus, a component correlated between xm and xm−1 cannot be subtracted anymore. Following this principle, a component parallel to each of x1…xM is subtracted from yn only once. By repeating this procedure for all vertices in ROI X, the signal components contained in ROI X will be subtracted from yn.
However, the orthogonalization method has ambiguity in its directionality. Because we cannot specify which cortical areas are truly or falsely informative before analysis, the directionality of orthogonalization cannot be determined. To deal with this situation, we can apply the orthogonalization bidirectionally (e.g., orthogonalize IT relative to V1 and vice versa) and compare which ROI better preserves information regarding the experimental conditions after orthogonalization.
We adopted this approach and first applied it to the simulation data. Specifically, we orthogonalized the estimated cortical current in IT relative to V1 and vice versa at 25–75 ms. The effects of orthogonalization were evaluated according to the correlation between the time series of estimated cortical current in V1 and the orthogonalized one in IT and vice versa. The estimated cortical current was averaged across trials for each artificial experimental condition, and correlations were calculated for all pairs of vertices between the two ROIs.
S11(A) Fig shows that the correlation between estimated cortical current in V1 and IT was significantly reduced after the orthogonalization for all tested MEG source estimation methods, irrespective of the directionality of the orthogonalization. These results suggest that orthogonalization of the time series of estimated cortical current removed contamination of the cortical current from other ROIs.
Then, we applied classification analysis to the orthogonalized estimated cortical current in V1 and IT at 50 ms, the middle of the corresponding time period. S11(B) Fig shows that prediction accuracy for both V1 and IT significantly dropped after the orthogonalization. These results suggest that orthogonalization can reduce the effects of information spreading but also ruins the information represented by patterns of estimated cortical current in truly informative ROIs.
Such information loss may come from inaccuracy of MEG source estimation. Because MEG source estimation is imperfect, estimated cortical current in IT may contain certain components that are correlated with source cortical current in both V1 and IT. Therefore, if V1 is orthogonalized relative to IT, estimated cortical current in V1 is recursively subtracted from itself. Such recursive subtraction may lead to the loss of informative components of source cortical current in V1, resulting in a significant drop in prediction accuracy.
To examine whether similar phenomena are observed in real data, we applied the same sequential orthogonalization procedure to the real data shown in Fig 7 and S10 Fig. We orthogonalized the estimated cortical current in HVC relative to that in V1 and vice versa. The orthogonalization was performed using a 50-ms time window, which was shifted with 50-ms steps to cover the whole time series.
The results were similar to those observed for the simulation. S11(C) Fig shows that the correlation between estimated cortical current in V1 and HVC significantly dropped after the orthogonalization. Prediction accuracy for the presented visual stimuli also significantly dropped to chance level at almost all times across both ROIs (S11(D) Fig). These results suggest that the method worked similarly: information loss caused by recursive subtraction also occurred in the real data. In addition, we confirmed that no significant difference was observed in the results presented in S11(D) Fig when the order of xm in Eq (B1) was randomly shuffled 10 times, indicating that the order of orthogonalization does not matter.
In summary, these results imply that although the orthogonalization methods can reduce the similarity of estimated cortical current in two ROIs, it could be difficult to reduce the effects of information spreading while preserving the information contents represented in the truly informative ROIs.
Supporting information
S1 Fig. Visual stimuli used for the MEG experiment.
(a) Pairs of wedges rotating clockwise in 30° steps. Black frames around the background are drawn for visibility (not shown in the actual experiment). (b) Correspondence between experimental conditions and labels. Three pairs of wedges presented in the upper right and lower left areas were labeled as condition 1, while the other three pairs of wedges presented in the upper left and lower right areas were labeled as condition 2.
https://doi.org/10.1371/journal.pone.0198806.s001
(PDF)
S2 Fig. Accuracy of source localization.
APRs for V1 at 50 ms and IT at 225 ms in experimental condition 2 (averaged across participants; error bars, s.d.; gray dashed lines, baseline value of APR) are shown. 0* indicates γ0 = 0.
https://doi.org/10.1371/journal.pone.0198806.s002
(PDF)
S3 Fig. Correlation coefficient between patterns of original and estimated cortical current.
Results for V1 at 50 ms and IT at 225 ms for artificial experimental condition 2 are shown (averaged across participants; error bars, s.d.; gray dashed lines, significance level [uncorrected P < 0.05]). 0* indicates γ0 = 0.
https://doi.org/10.1371/journal.pone.0198806.s003
(PDF)
S4 Fig. Time-resolved decoding in each ROI.
(a–d) Time courses of the prediction accuracy for MNE, MCE, VBMEG, and LCMV, respectively. Solid lines indicate the mean prediction accuracy across participants. Shading indicates the 1st–99th percentiles of prediction accuracy across participants. Red dashed lines indicate the mean significance levels across participants.
https://doi.org/10.1371/journal.pone.0198806.s004
(PDF)
S5 Fig. Time-resolved decoding in each ROI with random noise source in PR.
(a–d) Time courses of prediction accuracy for MNE, MCE, VBMEG, and LCMV, respectively. Solid lines indicate the mean prediction accuracy across participants. Shading indicates the 1st–99th percentiles of the prediction accuracy across participants. Results with random source in PR are shown in blue. Results corresponding to those shown in Fig 5B–5E are shown in black. Red dashed lines indicate the mean significance levels across participants.
https://doi.org/10.1371/journal.pone.0198806.s005
(PDF)
S6 Fig. Comparisons of difference in definition of statistical significance levels.
The number of time points that showed significant prediction accuracy via time-resolved decoding in each ROI (averaged across participants; error bars, s.d.). Results for the hyperparameters that achieved (a) the highest and (b) second-highest correlation coefficients in V1 (see Fig 4) are shown. No significant difference was observed among the definition of statistical significance levels (P > 0.05, Kruskal-Wallis test).
https://doi.org/10.1371/journal.pone.0198806.s006
(PDF)
S7 Fig. Spatial extent of information spreading.
(a, b) Results for V1 left and V1 right at 50 ms. (c, d) Results for IT left and IT right at 225 ms. The horizontal axes represent distance from the center of mass of each ROI. Solid lines indicate the mean prediction accuracy across participants. For an illustration purpose, prediction accuracy was averaged for each 5-mm distance bin. Shading indicates the 1st–99th percentiles of the prediction accuracy across participants. Each dashed line indicates the mean significance level across participants for each MEG source estimation method (they mostly overlap).
https://doi.org/10.1371/journal.pone.0198806.s007
(PDF)
S8 Fig. Spatial extent of t-value.
(a, b) Results for V1 left and V1 right at 50 ms. (c, d) Results for IT left and IT right at 225 ms. The horizontal axes represent distance from the center of mass of each ROI. Solid lines indicate the mean t-value across participants. For an illustration purpose, t-value was averaged for each 5-mm distance bin. Shading indicates the 1st–99th percentiles of the t-value across participants. Each dashed line indicates a significance level (t-test, uncorrected P < 0.05; degrees of freedom, 499).
https://doi.org/10.1371/journal.pone.0198806.s008
(PDF)
S9 Fig. Spatial extent of information spreading (univariate).
(a, b) Results for V1 left and V1 right at 50 ms. (c, d) Results for IT left and IT right at 225 ms. The horizontal axes represent distance from the center of mass of each ROI. Solid lines indicate the mean prediction accuracy across participants. For an illustration purpose, prediction accuracy was averaged for each 5-mm distance bin. Shading indicates the 1st–99th percentiles of the prediction accuracy across participants. Each dashed line indicates the mean significance level averaged across participants for each MEG source estimation method (they mostly overlap).
https://doi.org/10.1371/journal.pone.0198806.s009
(PDF)
S10 Fig. Information spreading in real data analysis.
(a) Time courses of prediction accuracy for each participant. Solid lines indicate the prediction accuracy for V1 (red) and HVC (blue), respectively. Gray dashed lines indicate significance levels. (b) Time courses of F-statistics for each participant, normalized between 0 and 1 for visibility. Vertical dashed lines indicate the onset latency of significant prediction accuracy for each ROI (red, V1; blue, HVC). (c) Maps of prediction accuracy obtained by searchlight decoding for each participant. Brain areas that showed significant prediction accuracy were colored. White and black enclosed areas are V1 and HVC, respectively.
https://doi.org/10.1371/journal.pone.0198806.s010
(PDF)
S11 Fig. Test of signal leakage reduction.
(a) Change of correlation by orthogonalization in simulation data (averaged across participants; error bars, s.d.). (b) Change of prediction accuracy by orthogonalization in simulation data (averaged across participants; error bars, s.d.;). (c) Change of correlation by orthogonalization in real data (averaged across participants; error bars, s.d.). Results of 51–100 ms and 101–150 ms are shown as representative examples. (d) Time courses of prediction accuracy after orthogonalization. Solid lines indicate the prediction accuracy for V1 (red) and HVC (blue) of each participant. Gray dashed lines indicate significance levels.
https://doi.org/10.1371/journal.pone.0198806.s011
(PDF)
S1 File. Individual data points from APR analysis (Fig 3 and S2 Fig).
https://doi.org/10.1371/journal.pone.0198806.s012
(XLSX)
S2 File. Individual data points from correlation analysis (Fig 4 and S3 Fig).
https://doi.org/10.1371/journal.pone.0198806.s013
(XLSX)
S3 File. Individual data points from time-resolved decoding in each ROI (Fig 5 and S4 Fig).
https://doi.org/10.1371/journal.pone.0198806.s014
(XLSX)
S4 File. Individual data points from searchlight decoding (Fig 6 and S7 Fig).
https://doi.org/10.1371/journal.pone.0198806.s015
(XLSX)
S5 File. Individual data points from real data analysis (Fig 7 and S9 Fig).
https://doi.org/10.1371/journal.pone.0198806.s016
(XLSX)
References
- 1. Sharon D, Hämäläinen MS, Tootell RBH, Halgren E, Belliveau JW. The advantage of combining MEG and EEG: Comparison to fMRI in focally stimulated visual cortex. Neuroimage. 2007;36: 1225–1235. pmid:17532230
- 2. de Peralta-Menendez RG, Gonzalez-Andino SL, Lutkenhoner B. Figures of merit to compare distributed linear inverse solutions. Brain Topogr. 1996;9: 117–124.
- 3. de Peralta-Menendez RG, Gonzalez-Andino SL, Grave de Peralta-Menendez R. A critical analysis of linear inverse solutions to the neuroelectromagnetic inverse problem. IEEE Trans Biomed Eng. 1998;45: 440–448. pmid:9556961
- 4. Molins A, Stufflebeam SM, Brown EN, Hämäläinen MS. Quantification of the benefit from integrating MEG and EEG data in minimum l2-norm estimation. Neuroimage. 2008;42: 1069–1077. pmid:18602485
- 5. Hauk O, Wakeman DG, Henson R. Comparison of noise-normalized minimum norm estimates for MEG analysis using multiple resolution metrics. Neuroimage. 2011;54: 1966–1974. pmid:20884360
- 6. Cottereau B, Jerbi K, Baillet S. Multiresolution imaging of MEG cortical sources using an explicit piecewise model. Neuroimage. 2007;38: 439–451. pmid:17889564
- 7. Grova C, Daunizeau J, Lina JM, Bénar CG, Benali H, Gotman J. Evaluation of EEG localization methods using realistic simulations of interictal spikes. Neuroimage. 2006;29: 734–753. pmid:16271483
- 8. Fukushima M, Yamashita O, Knösche TR, Sato M. MEG source reconstruction based on identification of directed source interactions on whole-brain anatomical networks. Neuroimage. The Authors; 2015;105: 408–427. pmid:25290887
- 9. Haxby J V, Gobbini MI, Furey ML, Ishai a, Schouten JL, Pietrini P. Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science (80-). 2001;293: 2425–30. pmid:11577229
- 10. Lemm S, Blankertz B, Dickhaus T, Müller KR. Introduction to machine learning for brain imaging. Neuroimage. Elsevier Inc.; 2011;56: 387–399. pmid:21172442
- 11. Contini EW, Wardle SG, Carlson TA. Decoding the time-course of object recognition in the human brain: From visual features to categorical decisions. Neuropsychologia. 2017;105: 165–176. pmid:28215698
- 12. Grootswagers T, Wardle SG, Carlson TA. Decoding Dynamic Brain Patterns from Evoked Responses: A Tutorial on Multivariate Pattern Analysis Applied to Time Series Neuroimaging Data. J Cogn Neurosci. 2017;29: 677–697. pmid:27779910
- 13. Pereira F, Mitchell T, Botvinick M. Machine learning classi!ers and fMRI: A tutorial overview. Neuroimage. 2009;45: S199–S209. pmid:19070668
- 14. Haynes JD. A Primer on Pattern-Based Approaches to fMRI: Principles, Pitfalls, and Perspectives. Neuron. Elsevier Inc.; 2015;87: 257–270. pmid:26182413
- 15. Carlson TA, Hogendoorn H, Kanai R, Mesik J, Turret J. High temporal resolution decoding of object position and category. J Vis. 2011;11: 9–9. pmid:21920851
- 16. Carlson T, Tovar DA, Alink A, Kriegeskorte N. Representational dynamics of object vision: The first 1000 ms. J Vis. 2013;13: 1–1. pmid:23908380
- 17. Clarke A, Devereux BJ, Randall B, Tyler LK. Predicting the Time Course of Individual Objects with MEG. Cereb Cortex. 2014; 1–11. pmid:25209607
- 18. Cox DD, Savoy RL. Functional magnetic resonance imaging (fMRI) “brain reading”: detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage. 2003;19: 261–270. pmid:12814577
- 19. Kamitani Y, Tong F. Decoding the visual and subjective contents of the human brain. Nat Neurosci. 2005;8: 679–85. pmid:15852014
- 20. Miyawaki Y, Uchida H, Yamashita O, Sato M, Morito Y, Tanabe HC, et al. Visual Image Reconstruction from Human Brain Activity using a Combination of Multiscale Local Image Decoders. Neuron. 2008;60: 915–929. pmid:19081384
- 21. van de Nieuwenhuijzen ME, Backus AR, Bahramisharif A, Doeller CF, Jensen O, van Gerven MAJ. MEG-based decoding of the spatiotemporal dynamics of visual category perception. Neuroimage. 2013;83: 1063–1073. pmid:23927900
- 22. Simanova I, van Gerven MAJ, Oostenveld R, Hagoort P. Predicting the Semantic Category of Internally Generated Words from Neuromagnetic Recordings. J Cogn Neurosci. 2015;27: 35–45. pmid:25061927
- 23. Ohata R, Ogawa K, Imamizu H. Single-trial prediction of reaction time variability from MEG brain activity. Sci Rep. 2016;6: 27416. pmid:27250872
- 24. Ramkumar P, Hansen BC, Pannasch S, Loschky LC. Visual information representation and rapid scene categorization are simultaneous across cortex: An MEG study. Neuroimage. 2016; pmid:27001497
- 25. Tadel F, Baillet S, Mosher JC, Pantazis D, Leahy RM. Brainstorm: A user-friendly application for MEG/EEG analysis. Comput Intell Neurosci. 2011;2011. pmid:21584256
- 26. Fonov V, Evans AC, Botteron K, Almli CR, McKinstry RC, Collins DL. Unbiased average age-appropriate atlases for pediatric studies. Neuroimage. 2011;54: 313–327. pmid:20656036
- 27. Kybic J, Clerc M, Abboud T, Faugeras O, Keriven R, Papadopoulo T. A common formalism for the integral formulations of the forward EEG problem. IEEE Trans Med Imaging. 2005;24: 12–28. pmid:15638183
- 28. Gramfort A, Papadopoulo T, Olivi E, Clerc M. OpenMEEG: opensource software for quasistatic bioelectromagnetics. Biomed Eng Online. 2010;9: 45. pmid:20819204
- 29. Hämäläinen MS, Ilmoniemi RJ. Interpreting magnetic fields of the brain: minimum norm estimates. Med Biol Eng Comput. 1994;32: 35–42. pmid:8182960
- 30. Matsuura K, Okabe Y. Selective minimum-norm solution of the biomagnetic inverse problem. IEEE Trans Biomed Eng. 1995;42: 608–15. pmid:7790017
- 31. Uutela K, Hamalainen MS, Somersalo E. Visualization of magnetoencephalograpic data using minimum current estimates. Neuroimage. 1999;10: 173–180. pmid:10417249
- 32. Tibshirani R. Regression Shrinkage and Selection via the Lasso. Journal of the Royal Statistical Society B. 1996. pp. 267–288.
- 33.
Ghaoui L El, Viallon V. Safe feature elimination for the LASSO and sparse supervised learning problems. EECS Tech Rep. 2010;
- 34. Kwangmoo K, Seung-Jean K, Stephen B. An Interior-Point Method for Large-Scale L1-Regularized Logistic Regression. J Mach Learn Res. 2007;8: 1519–1555.
- 35. Kajihara S, Ohtani Y, Goda N, Tanigawa M, Ejima Y, Toyama K. Wiener filter-magnetoencephalography of visual cortical activity. Brain Topogr. 2004;17: 13–25. pmid:15669752
- 36. Sato M, Yoshioka T, Kajihara S, Toyama K, Goda N, Doya K, et al. Hierarchical Bayesian estimation for MEG inverse problem. Neuroimage. 2004;23: 806–26. pmid:15528082
- 37. MacKay DJC. Bayesian Interpolation. Neural Comput. 1992;4: 415–447.
- 38.
Neal RM. Bayesian Learning for Neural Networks. NewYork: Springer-Verlag; 1996.
- 39. Van Veen B, van Drongelen W, Yuchtman M, Suzuki A. Localization of brain electrical activity via linearly constrained minimum variance spatial filtering. IEEE Trans Biomed Eng. 1997;44: 867–880. pmid:9282479
- 40. Hanley JA, McNeil BJ. The Meaning and Use of the Area under a Receiver Operating (ROC) Curvel Characteristic. Radiology. 1982;143: 29–36. pmid:7063747
- 41. He H, Garcia EA. Learning from imbalanced data. IEEE Trans Knowl Data Eng. 2009;21: 1263–1284.
- 42.
Davis J, Goadrich M. The Relationship Between Precision-Recall and ROC Curves. Proc 23rd Int Conf Mach Learn—ICML’06. 2006; 233–240. https://doi.org/10.1145/1143844.1143874
- 43. Saito T, Rehmsmeier M. The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLoS One. 2015;10: 1–21. pmid:25738806
- 44. Cortes C, Vapnik V. Support-vector networks. Mach Learn. 1995;20: 273–297.
- 45. Cichy RM, Pantazis D, Oliva A. Resolving human object recognition in space and time. Nat Neurosci. 2014;17: 455–62. pmid:24464044
- 46. Kriegeskorte N, Goebel R, Bandettini P. Information-based functional brain mapping. Proc Natl Acad Sci U S A. 2006;103: 3863–3868. pmid:16537458
- 47. de Cheveigné A, Simon JZ. Denoising based on time-shift PCA. J Neurosci Methods. 2007;165: 297–305. pmid:17624443
- 48. Epstein R, Kanwisher N. A cortical representation of the local visual environment. Nature. 1998;392: 598–601. pmid:9560155
- 49. Kourtzi Z, Kanwisher N. Cortical regions involved in perceiving object shape. J Neurosci. 2000;20: 3310–3318. pmid:10777794
- 50. Kanwisher N, McDermott J, Chun MM. The fusiform face area: a module in human extrastriate cortex specialized for face perception. J Neurosci. 1997;17: 4302–11. pmid:9151747
- 51. Sereno MI, Dale AM, Reppas JB, Kwong KK, Belliveau JW, Brady TJ, et al. Borders of multiple visual areas in humans revealed by functional magnetic resonance imaging. Science. 1995;268: 889–893. pmid:7754376
- 52. Yoshor D, Bosking WH, Ghose GM, Maunsell JHR, Hall J, Street C. Receptive fields in human visual cortex mapped with surface electrodes. Cereb Cortex. 2007;17: 2293–2302. pmid:17172632
- 53. Schmolesky MT, Wang Y, Hanes D, Thompson KG, Leutgeb S, Schall JD, et al. Signal timing across the macaque visual system. J neurophy. 1998;79: 3272–3278.
- 54. Zou H, Hastie T. Regularization and variable selection via the elastic-net. J R Stat Soc. 2005;67: 301–320.
- 55. Glasser MF, Coalson TS, Robinson EC, Hacker CD, Harwell J, Yacoub E, et al. A multi-modal parcellation of human cerebral cortex. Nature. 2016;536: 171–8. pmid:27437579
- 56. Gohel B, Lim S, Kim MY, Kwon H, Kim K. Dynamic pattern decoding of source-reconstructed MEG or EEG data: Perspective of multivariate pattern analysis and signal leakage. Comput Biol Med. Elsevier Ltd; 2018;93: 106–116. pmid:29291534
- 57. Kriegeskorte N, Kievit RA. Representational geometry: integrating cognition, computation, and the brain. Trends Cogn Sci. 2013; 1–12. pmid:23876494
- 58. Clarke A, Tyler LK. Object-specific semantic coding in human perirhinal cortex. J Neurosci. 2014;34: 4766–75. pmid:24695697
- 59. Kriegeskorte N, Mur M, Ruff D, Kiani R. Matching categorical object representations in inferior temporal cortex of man and monkey. Neuron. 2008;60: 1126–1141. pmid:19109916
- 60. Wardle SG, Kriegeskorte N, Grootswagers T, Khaligh-Razavi SM, Carlson TA. Perceptual similarity of visual patterns predicts dynamic neural activation patterns measured with MEG. Neuroimage. 2016;132: 59–70. pmid:26899210
- 61. Cichy RM, Khosla A, Pantazis D, Torralba A, Oliva A. Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence. Sci Rep. 2016;6: 27755. pmid:27282108
- 62.
Sato M, Miyawaki Y. Spatial spreading of representational geometry through source estimation of magnetoencephalography signals. 2017 Int Work Pattern Recognit Neuroimaging, PRNI 2017. 2017; https://doi.org/10.1109/PRNI.2017.7981509
- 63. Brookes MJ, Woolrich MW, Barnes GR. Measuring functional connectivity in MEG: A multivariate approach insensitive to linear source leakage. Neuroimage. Elsevier Inc.; 2012;63: 910–920. pmid:22484306
- 64. Hipp JF, Hawellek DJ, Corbetta M, Siegel M, Engel AK. Large-scale cortical correlation structure of spontaneous oscillatory activity. Nat Neurosci. 2012;15: 884–890. pmid:22561454
- 65. Colclough GL, Brookes MJ, Smith SM, Woolrich MW. A Symmetric Multivariate Leakage Correction for MEG Connectomes. Neuroimage. 2015;117: 439–448. pmid:25862259