Figure 1.
Visual support matrix displayed on a computer screen to the participant during the auditory P300-BCI experiment [22].
The matrix was identical to the visual P300 BCI matrix. The speakers displayed at the top left corner of the matrix indicate the auditory presentation of numbers and were not displayed during the actual experiment.
Figure 2.
Every participant was presented with an auditory oddball, and performed a visual and auditory P300 BCI session.
For both visual and auditory P300 BCI online feedback was provided. Performance was reevaluated offline by calculating the number of repetitions per stimulus the user needed to reach 70% accuracy. This performance measure was correlated with features extracted from the auditory oddball session to assess whether it can be used to predict BCI aptitude.
Figure 3.
Probability of selecting the target matrix element (center) or a matrix element around the target is color coded on a logarithmic scale.
The x-axis shows how many columns to the left (negative) or to the right (positive) an error occurs. Correspondingly, the y-axis shows the probabilities for errors for rows above or below the target. Both in the visual P300 BCI (left) and the auditory P300 BCI (right) errors occur with a much higher probability on the same row or column as the target. This unequal distribution of the error probability was the motivation for applying mutual information to measure bitrate.
Figure 4.
Performance distributions for auditory P300 BCI (left) and visual P300 BCI (right). The median is indicated by a vertical black line.
Figure 5.
The letter selection accuracy is plotted as a function of the number of stimulus repetitions, i.e. flashes or spoken row/column numbers.
Data from all 63 EEG channels and a time window of 800 ms was available for the classifier. It can be seen that in the auditory modality only high aptitude users achieve an error rate below 20% comparable to the visual modality for all users. Dashed lines: visual P300 BCI; Continuous lines: auditory P300 BCI.
Figure 6.
The letter selection accuracy is plotted as a function of time.
The data was split into non-overlapping 50 ms time bins that were used to train and test the classifier. Data from all 63 EEG channels was available for the classifier. For the visual P300 BCI the highest accuracy occurs in the expected P300 time window. In the auditory BCI neither the high nor the low aptitude users achieve an accuracy as high as in the visual P300 BCI. Dashed lines: visual P300 BCI; continuous lines: auditory P300 BCI.
Figure 7.
Responses to auditory oddball (A), visual P300 BCI (B) and auditory P300 BCI (C) are shown from left to right.
Top row: average amplitude of the full spatio-temporal feature matrix of the target non-target difference for each experiment. Middle row: time course at Cz (auditory oddball and visual P300 BCI) and Pz (auditory P300 BCI) of the averaged ERP for targets (continuous lines) and non-targets (dashed lines). Bottom row: topographic distribution of the target non-target difference at 200, 300 and 400 ms. For the auditory oddball, subjects were split in high and low aptitude users at the median of the mean performance in the auditory and visual P300 BCI.
Figure 8.
Signed values between auditory oddball amplitudes of all time points and channels with auditory P300 BCI performance (defined as the number of sequences needed to reach 70% accuracy) are shown in red for positive correlations and in blue for negative correlations (A).
Two elements from the matrix were selected for visualization using scatter plots (B) showing a correlation of on electrode FC5 and a correlation of
on electrode PO2. Topographic distributions of the signed
values are shown at the bottom (C). Note that due to the use of “number of sequences needed to reach 70% accuracy” as performance measure positive correlations indicate a decrease in performance with increasing amplitude, whereas negative correlations indicate an increase of performance with decreasing amplitude.
Figure 9.
Signed values between auditory oddball amplitudes of all time points and channels with visual P300 BCI performance (defined as the number of sequences needed to reach 70% accuracy) are shown in red for positive correlations and in blue for negative correlations (A).
Two elements from the matrix were selected for visualization using scatter plots (B) showing a correlation between performance and amplitude of on electrode C2 and of
on electrode PO7. Topographic distributions of the signed
values are shown at the bottom (C). Note that due to the use of “number of sequences needed to reach 70% accuracy” as performance measure positive correlations indicate a decrease in performance with increasing amplitude, whereas negative correlations indicate an increase of performance with decreasing amplitude.
Table 1.
Amplitudes, latencies and correlations thereof with BCI performance shown for N200 (minimal amplitude before latency of P300 on Cz), P300 (maximum between 250 and 700 ms on Cz) and late ERP component (maximum after P300 latency on POz).