Fig 1.
Arrangement of EEG channels (blue) and fNIRS optodes (red: Sources, green: Detectors) on the frontal (left) and motor (right) areas. Channels were numbered in the same manner as in a previous study [20].
Fig 2.
Illustration of a single trial of the experiment.
Each trial consisted of an introduction period of 2 s, a task period of 10 s, and an inter-trial rest period (stop and rest) of 16–18 s. During the introduction period, a random task (MA, MI, or IS) was displayed to the participant. After a short beep, the participant performed the task displayed in the introduction period while looking at a fixation cross. When a STOP sign was displayed with a second short beep, the participants stopped performing the task and relaxed during the random-length inter-trial rest period.
Fig 3.
CSP and sLDA stand for common spatial pattern and shrinkage linear discriminant analysis, respectively. To perform meta-classification, we concatenated the outputs of individual EEG and fNIRS classifiers to construct feature vectors for the meta-classifier. The “one-versus-one” block represents the strategy used to solve the three-class classification problem by dividing it into three binary classification problems and employing majority voting (VOTE) based on the result of each binary classification to predict the class.
Fig 4.
(a) MI vs. IS EEG classification accuracies as a function of the number of EEG channels. Green bars indicate the grand average classification accuracies calculated using the sequential backward selection algorithm. The color map indicates the statistical significance between the differences in classification accuracy calculated according to the number of EEG channels (*p < 0.05). (b) MI vs. IS EEG classification accuracies calculated using the four optimal configurations with two EEG channel, which were (c) (Cz, CP3; the best), (d) (Cz, C3), (e) (CP3, CP4), and (f) (C3, C4; the fourth best). The red and green horizontal dashed lines indicate the threshold for an effective BCI (70%) and the value corresponding to the two EEG channels (x-axis) shown in (a), respectively. The error bars indicate standard deviation.
Fig 5.
(a) Highest MA vs. IS fNIRS classification accuracies as a function of the number of SD pairs shown in (b). Statistical significance was calculated between the classification accuracies achieved using all the SD pairs and various numbers of SD pairs (*p < 0.05, **p < 0.01, ***p < 0.001). The error bars represent the standard deviation.
Table 1.
Individual three-class hBCI classification accuracies.