Brain response to luminance-based and motion-based stimulation using inter-modulation frequencies

Xin Zhang; Guanghua Xu; Jun Xie; Xun Zhang

doi:10.1371/journal.pone.0188073

Abstract

Steady state visual evoked potential (SSVEP)-based brain computer interface (BCI) has advantages of high information transfer rate (ITR), less electrodes and little training. So it has been widely investigated. However, the available stimulus frequencies are limited by brain responses. Simultaneous modulation of stimulus luminance is a novel method to resolve this problem. In this study, three experiments were devised to gain a deeper understanding of the brain response to the stimulation using inter-modulation frequencies. First, luminance-based stimulation using one to five inter-modulation frequencies was analyzed for the first time. The characteristics of the brain responses to the proposed stimulation were reported. Second, the motion-based stimulation with equal luminance using inter-modulation frequencies was also proposed for the first time. The response of the brain under these conditions were similar to that of luminance-based stimulation which can induce combination frequencies. And an elementary analysis was conducted to explain the reason of the occurrence of combination frequencies. Finally, the online test demonstrated the efficacy of our proposed two stimulation methods for BCI. The average ITRs reached 34.7836 bits/min and 39.2856 bits/min for luminance-based and motion-based stimulation respectively. This study demonstrated that the simultaneous modulation of stimulus luminance could extend to at least five frequencies to induce SSVEP and the brain response to the stimulus still maintained a certain positive correlation with luminance. And not only luminance-based stimulation, but also motion-based stimulation with equal luminance can elicit inter-modulation frequencies to effectively increase the number of targets for multi-class SSVEP.

Citation: Zhang X, Xu G, Xie J, Zhang X (2017) Brain response to luminance-based and motion-based stimulation using inter-modulation frequencies. PLoS ONE 12(11): e0188073. https://doi.org/10.1371/journal.pone.0188073

Editor: Huiguang He, Chinese Academy of Sciences, CHINA

Received: June 27, 2017; Accepted: October 31, 2017; Published: November 15, 2017

Copyright: © 2017 Zhang et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within the paper and its Supporting Information files.

Funding: This work was supported by the Integration Project of Major Research plan of the Natural Science Foundation of China (91420301), the National Natural Science Foundation of China (no. 51475360) and the science and technology plan project of Guangdong province under grant no. 2015B020214001. No additional external funding was received for this study. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Introduction

Steady-state visual evoked potentials (SSVEP) are induced by visual stimulation containing a constant flickering frequency [1] ranging from 1 to 100 Hz [2]. The low frequency bands (5–12 Hz), middle frequency bands(12–30 Hz) and high frequency bands (30–60 Hz) are the three main frequency ranges. In the low and medium frequency bands, SSVEPs at 15 Hz show the largest amplitude [3]. Hence many SSVEP brain computer interface (BCI) systems select frequencies in the low and middle frequency bands as visual stimulation frequencies [4, 5]. However, visual stimulation at low frequencies can induce seizures and easily to cause eye discomfort. Furthermore, the low frequency bands overlap with EEG in the alpha band which may increase the difficulty of identification. Hence several research teams use frequencies in the high frequency bands as visual stimulation frequencies in SSVEP-BCI systems to avoid the risk of seizures and slow visual fatigue [6, 7]. However, accuracy and information transfer rate (ITR) are significantly lower due to lower amplitudes induced by high frequency stimulus [7]. What's more, limited by the screen refresh rate, cathode ray tube (CRT) and liquid crystal display (LCD) monitors can only produce limited available frequencies. These frequencies must be divided exactly by the screen refresh rate. What's more, single frequency stimulus may induce harmonic frequencies in SSVEP. That is to say, the fundamental frequency and the harmonic frequencies cannot be the stimulus frequency at the same time. To sum up, the effective visual stimulation frequencies are limited in SSVEP-BICs.

Recently, several studies have attempted to increase the number of targets by using limited frequencies. One method uses the phase information as using different phase lags can make different targets even using the same frequencies [8]. However, the number of phase coded targets is still limited by the refresh rate. Another method is coding multiple different frequencies into one target [9–12]. There are two ways to realize this coding method: sequential multiple frequencies flickering method and simultaneous multiple frequencies protocol. As to sequential multiple frequencies flickering method, the length of the stimulation time will increase as the number of frequencies is increased. Due to the shortages mentioned above, the simultaneous multiple frequencies protocol is proposed. Most studies use an LCD monitor or LEDs to present visual stimulation that combines two frequencies. However, in those studies, two LEDs are used to produce the dual frequency visual stimulation so that participants are required to focus their eyes on the middle of the two flickering points [11]. Otherwise, a shift of attention may be occur which may cause some frequency-peaks to be lost. Moreover, most studies have reported the characteristics of EEG response to the dual inter-modulation frequency visual stimulation. To gain a comprehensive understanding of luminance-based inter-modulation, the characteristics of EEG response to multiple inter-modulation frequencies visual stimulus is explored in this study.

SSVEPs mainly occur in the primary visual cortex from which there are two pathways to output information: ventral pathway and dorsal pathway [13]. The ventral pathway, also known as the P-pathway, is a color-sense-associated and object recognition pathway, which detects luminance and color. The dorsal pathway, also known as the M-pathway, is associated with spatial processing and motion recognition, and detects motion speed and direction [14, 15]. The existing studies on simultaneous multiple frequencies protocol are mostly based on changes of luminance and color [16] which induce potentials in the ventral pathway. The characteristics of steady-state motion visual evoked potential (SSMVEP) [17] in the dorsal pathway using simultaneous multiple frequencies protocol is still unknown.

In this study, two novel stimulation methods using inter-modulation frequencies are proposed. The first method is simultaneous modulation of stimulus luminance which can produce multiple inter-modulation frequencies using a single LED. Some interesting discoveries are shown in this study towards a comprehensive understanding of luminance-based inter-modulation. The other method is simultaneous modulation of stimulus motion which maintains equal luminance. The difference of the characteristics in the brain response between these two methods is discussed. Besides, both the methods mix multiple frequencies into one single target thus participants no longer need to shift their attention. Furthermore, an online test is implemented to demonstrate the feasibility of these two methods for SSVEP-BCIs. The results show that the brain response to stimulation using multiple inter-modulation frequencies still maintains a certain positive correlation with luminance and that both proposed methods can encode more target stimuli with limited frequencies.

Materials and methods

Ethics statement

Subjects were studied after giving informed written consent in accordance with a protocol approved by the institutional review board of Xi’an Jiaotong University.

Design of luminance-based stimulation

In this study, a green straw hat LED is chosen as the visual stimulation. To avoid eye discomfort caused by the bright light from the middle of the LED, a hemispherical lampshade made of light diffusing material with 85% light transmittance is used. The light transmittance through the lampshade from the fluorescent lamp ranges from 80% to 90%. And 85% light transmittance is a balance between the participants' visual comfort and the intensity of the stimulus. What's more, the diameter of the hemispherical lampshade is 56mm which is just in the commonly used standard size. And the viewing distance is 70 cm in commonly studies. So the visual angle is 4.6 degrees which is a suitable visual angle for subjects. The stm32F103RB (STMicroelectronics company) is chosen as the microcontroller to produce multiple frequencies. By using the timer of STM32, the pulse width modulation (PWM) signal can be produced. Changing the duty ratio of the PWM signal changes the luminance of the LED. If the duty ratio of the PWM changes as DR (see formula (1)), the multi-frequency stimulation is possible. Besides, the timer is also used to guarantee the accuracy of the cycle. The time interval is 1 millisecond (ms). (1) where f₁,f₂,…,f_n are frequencies of the stimulation, A₁,A₂,…,A_n are amplitudes, n is the number of the multiple frequencies and t is time interval.

To guarantee the proposed method could produce the desired simultaneous flickering frequencies, the following test is performed prior to the experiments. Photistor (3DU5C) is chosen as the sensor to measure the change of light intensity of LED flickering. A resistor was connected to the photistor and a 3.3V power supply was used. The change in the resistance voltage was acquired by the data acquisition card NI9234 (National Instruments, USA). LED and photistor were placed face to face. The distance between them was 6 centimeter (cm). The simultaneous flickering frequencies were 16Hz, 15Hz, 14Hz, 13Hz and 12Hz. Fig 1 shows the result of the test. Fig 1A shows the simulation of duty ratio of PWM. Fig 1C shows the signal acquired by NI9234. Fig 1A and 1C are almost same. Their spectrums also have the same peak frequencies (Fig 1B and 1D respectively).

Download:

Fig 1. Luminance-based stimulation with 16Hz, 15Hz, 14Hz, 13Hz and 12Hz simultaneously.

(a) duty ratio of the PWM (b) spectrum of the duty ratio (c) light intensity of LED flickering (d) spectrum of light intensity.

https://doi.org/10.1371/journal.pone.0188073.g001

Design of motion-based stimulation

The ring chessboard is chosen as the basic pattern of the proposed motion-based stimulation. The ring chessboard paradigm is a development of Newton’s rings paradigm [17] which can provide a comparable performance with low-adaptation characteristic and less visual discomfort for BCI applications. In the ring chessboard shown in Fig 2, each ring is divided into black and white lattices of equal numbers and sizes, so the area of the bright and dark areas in each ring are always equal. The brightness value of the central part of the stimulus unit is always set to the background brightness, and the area of the central part remains unchanged. As such, the luminance of the designed motion-based stimulation remains unchanged. The S1 File in the Supporting Information shows the main coding algorithms for producing the stimulation with dual frequencies (the style of the paradigm needs users to create it by themselves). The method to produce the chessboard paradigm proposed in this study is described below.

Download:

Fig 2. Motion-based stimulation.

https://doi.org/10.1371/journal.pone.0188073.g002

The stimulus program is developed under MATLAB using the Psychophysics Toolbox. Formula for generating pattern of ring checkerboard stimulating unit is as follows. (2) where φ(t) is the phase value function in contraction expansion, N_c is the number of white and black squares contained in a single ring, L₀ is background brightness, W is the width of the ring and A is the amplitude of the motion.

The motion reversal process of the motion-based stimulation is controlled by phase functions φ(t). (3) where f₁,f₂,…,f_n are frequencies of the stimulation. The ring chessboard will contract as the phase changes from 0 to pi and it will expand as the phase changes from pi to 0 (The specific forms of movement can refer to the S1 Video). Motion direction changes twice in one cycle. The motion inversion frequency is defined as the frequency of motion direction changes. And the motion inversion frequency is twice of motion frequency. The annular expansion and contraction of the ring chessboard forms visual stimuli with inter-modulation frequencies and the luminance remains unchanged.

Experimental design

Ten healthy male subjects (ages 23–25) participated in the experiments. They were all volunteers from Xi'an Jiaotong University. EEG signals were recorded from six EEG electrodes (G.USBamp, G.tec Guger Technologies, Austria). Based on the international 10–20 system, the six EEG electrodes were placed at PO3, PO4, POz, O1, O2, Oz. The unilateral (left or right) earlobe was used as the recording reference and Fpz was used as ground. All electrodes' impedances were kept below 5 kOhm. The sampling frequency was 1200 Hz. Band-passed filter between 2 and 100 Hz and notch filter from 48Hz to 52Hz were used to remove artifacts and power line interface when acquiring EEG signals. The subjects were asked to sit on a comfortable chair in a quiet and ordinarily lit office room. All of the subjects had undergone BCIs before and had normal eyesight. Experiments were designed as described in Table 1.

Download:

Table 1. Experimental design.

https://doi.org/10.1371/journal.pone.0188073.t001

E1: Offline experiment of brain response under luminance-based stimulation using inter-modulation frequencies.

There were two tasks during this experiment: equal amplitude stimuli using multiple inter-modulation frequencies and unequal amplitude stimuli using fixed dual inter-modulation frequencies. The green LED with a hemispherical lampshade was placed 70 cm in front of the subject. The LED would flicker for 5 s and light out for 5 s automatically. All the subjects were required to stare at the flickering LED for 5 s per trial, 5 trials each inter-modulation frequency, with an interval of 5 s. Then the inter-modulation frequencies of the flickering LED would change as described below.

Task 1: equal amplitude stimuli using multiple inter-modulation frequencies. There were five different kinds of inter-modulation frequencies in this task including 16 Hz, 16+15 Hz, 16+15+14 Hz, 16+15+14+13 Hz and 16+15+14+13+12 Hz. The plus meant simultaneous multiple frequencies as formula (1) shows. The amplitudes of each frequencies were the same and the sum of the amplitudes was 450. As to 16+15+14+13+12 Hz, it means f₁ = 16Hz, f₂ = 15Hz, f₃ = 14Hz, f₄ = 13Hz, f₅ = 12Hz and A₁ = A₂ = A₃ = A₄ = A₅ = 90.

Task 2: unequal amplitude stimuli using fixed dual inter-modulation frequencies. There were also five different kinds of inter-modulation frequencies in this task. The frequencies were 16+15 Hz. But the amplitude of each frequency was different. The amplitudes were as follows: (A₁ = 111, A₂ = 333), (A₁ = 150, A₂ = 300), (A₁ = 225, A₂ = 225), (A₁ = 300, A₂ = 150), (A₁ = 333, A₂ = 111).

During all experiments, the EEG signals were recorded in the computer hard disk automatically.

E2: Offline experiment of comparisons between luminance-based and motion-based stimulation using dual inter-modulation frequencies.

Luminance-based stimulation and motion-based stimulation described before were all used in this experiment. The inter-modulation frequencies were 5+8 Hz. All the subjects were required to stare at the two stimulations for 10 s per trial, with five trials for each stimulation, with an interval of 5 s respectively. As to luminance-based stimulation, the green LED with a hemispherical lampshade was placed 70 cm in front of the subject. And the LED would flicker for 10 s and then turned off for 5 s automatically. As for motion-based stimulation, it was presented in the middle of a LCD monitor. The diameter of the ring chessboard was 60 mm. The ring chessboard would for 10 s and then disappeared for 5 s automatically too. EEG signals were recorded in the computer hard disk automatically.

E3: Online experiment of luminance-based and motion-based stimulation using inter-modulation frequencies.

In regard to luminance-based stimulation, there were ten targets with dual inter-modulation frequencies including 16+15 Hz, 16+14 Hz, 16+13 Hz, 16+17 Hz, 15+14 Hz, 15+13 Hz, 15+17 Hz, 14+13 Hz, 14+17 Hz, 13+17 Hz. As for motion-based stimulation, there were fifteen targets with frequencies including 17.1+17.1 Hz, 17.1+15 Hz, 17.1+13.3 Hz, 17.1+12 Hz, 17.1+20 Hz, 15+15 Hz, 15+13.3 Hz, 15+12 Hz, 15+20 Hz, 13.3+13.3 Hz, 13.3+12 Hz, 13.3+20 Hz, 12+12 Hz, 12+20 Hz, 20+20 Hz. The fifteen targets were presented on the monitor simultaneously. There were three rows and five targets in each row. The stimulation duration per trial was divided into six levels, from 1 s to 6 s with an interval of 1 s. Subjects were required to stare on the targets one by one. The luminance-based experiments were carried out for 5 runs and the motion-based experiments were carried out for 6 runs. And the identification results would be shown to subjects automatically during the interval.

Data analysis

Canonical correlation analysis.

Canonical Correlation Analysis (CCA) is widely used in SSVEP target recognition. It is carried out to calculate the correlations between reference signals and multi-channel EEG data [18]. The formula to calculate correlation coefficient ρ is as follows. (4) where X is the EEG data, Y is the reference signals.

In this study, the reference signal Y₁ in the luminance-based online experiment and the reference signal Y₂ in the motion-based online experiment were chosen as follows: (5) where Fs is the sampling frequency and f₁,f₂ were the inter-modulation frequencies. Then the maximum of canonical coefficients was considered as the focused target. Besides, due to the latency delay in the visual system based on prior research [19], the date epochs began to be processed at 0.14 s.

CCA-based spatial filter.

To enhance the signal-to-noise ratio, the CCA-based spatial filter was used to analyze the offline experimental data. When the correlations were calculated as in formula (4), the weight vectors w_x could be obtained at the same time. The multi channel EEG data can be converted into one-dimensional signals by multiplying the weight vectors w_x as follows.

(6)

Power spectrum density.

The peak values at the stimulation frequencies in the power spectrum density (PSD) reflect the effect of inducing SSVEP by the designed stimulus. Hence the power spectrum density was chosen as the method to analyze the characteristics of the EEG response to multiple inter-modulation frequencies visual stimulus.

In this study, the Welch power spectrum density was used to estimate signals by dividing a datum with a length of N into M segments. The formula is as follows.

(7)

Statistical analysis.

Statistical analysis is conducted using repeated measures analysis of variance with Bonferroni posthoc analysis. Statistical significance is defined as p < 0.05.

Results

Offline experimental results of brain response under luminance-based stimulation using inter-modulation frequencies

Fig 3 shows the time domain waveforms, PSD and time-frequency maps acquired while a participant was staring at the equal amplitude stimulation using multiple inter-modulation frequencies as described before. It was clearly observed from the PSD that the peaks were evoked at the frequencies of the stimuli. However, previous studies [20–22] showed that it would evoke the peaks at m × f₁ + n × f_h(m,n = 0,±1,±2…) where f₁ and f_h were the dual inter-modulation frequencies. So as for the stimulus with 16+15+14 Hz, the peak at 14 Hz in the PSD may have been evoked by 14 Hz in the stimulus or in the combination frequency (14 = 2 × 15–16). So ANOVA on the peak values in the PSD was conducted to distinguish the difference. Table 2 shows the P values for all the subjects. Take subject 1 as an example, there was a significant difference on the peak values at 15 Hz in the PSD between the subject staring at the stimulation with 16 Hz and the stimulation with 16+15 Hz (P = 0.005<0.05). Similarly, there was significant difference on the peak values at 14 Hz in the PSD between the subject staring at the stimulation with 16+15 Hz and the stimulation with 16+15+14 Hz (P = 0.004<0.05). And there was a significant difference on the peak values at 13 Hz in the PSD between the subject staring at the stimulation with 16+15+14 Hz and the stimulation with 16+15+14+13 Hz (P = 0.029<0.05). Furthermore, there was a significant difference on the peak values at 12 Hz in the PSD between the subject staring at the stimulation with 16+15+14+13 Hz and the stimulation with 16+15+14+13+12 Hz (P = 0.04<0.05). The above mentioned results demonstrated that the stimulation would induce the corresponding frequencies in SSVEP when increasing frequencies in the inter-modulation frequencies stimulation and that the newly added induced frequencies in SSVEP were mainly caused by the added frequencies in the stimulation and not by the combination frequencies.

Download:

Fig 3. The time domain waveforms, power spectrum density and time-frequency maps acquired while a participant was staring at the equal amplitude stimulations.

(a) The time domain waveforms. (b) Power spectrum density. (c) Time-frequency maps.

https://doi.org/10.1371/journal.pone.0188073.g003

Download:

Table 2. ANOVA analyses on the peak values at the corresponding frequencies in the PSD while subjects stared at the stimulations with different kinds of frequencies.

https://doi.org/10.1371/journal.pone.0188073.t002

Even though the amplitudes of the stimulus at each frequency were equal, the peak values at the corresponding frequencies in PSD evoked by the stimulus were not the same. There was an interesting discovery in this result. The sum of PSD at the stimulus frequencies was calculated. Fig 4 shows the variance maps of the sum of PSD at the stimulus frequencies. The '1' in x axis indicates the sum of peak values at 16 Hz in PSD. The '2' in x axis indicates the sum of peak values at 16 Hz and 15 Hz in PSD. The '3' in x axis indicates the sum of peak values at 16 Hz, 15 Hz and 14 Hz in PSD. The '4' in x axis indicates the sum of peak values at 16 Hz, 15 Hz, 14 Hz and 13 Hz in PSD. The '5' in x axis indicates the sum of peak values at 16 Hz, 15 Hz, 14 Hz, 13 Hz and 12 Hz in PSD. The results showed that there were no significant difference between the different sums of PSD for each subject (P>0.05). That meant the sum of the peak values at the multiple inter-modulation frequencies in the PSD demonstrated no significant difference.

Download:

Fig 4. The variance maps about the sum of the peak values at the stimulus frequencies in the PSD.

https://doi.org/10.1371/journal.pone.0188073.g004

Fig 5 shows the mean of the peak values at 15 Hz in the PSD among all the participants and the mean of the peak values at 16 Hz in the PSD among all the participants while they were staring at the unequal amplitude stimulation using dual-modulation frequencies as described before. As the increase of one amplitude of the stimulation using dual inter-modulation frequencies, the peak values at corresponding frequency in the PSD increased. However the sum of the peak values at 15 Hz and 16 Hz in the PSD had no significant difference (P = 0.999>0.05) as shown in Fig 6. As such, it was concluded that the sum of the peak values at the two frequencies in the PSD had no significant difference as to unequal amplitude stimuli using fixed dual inter-modulation frequencies. That's to say the brain response had a positive correlation with the light intensity of the stimulation using inter-modulation frequencies.

Download:

Fig 5. The sum of peak values at 15 Hz in the PSD among all the participants and peak values at 16 Hz in the PSD among all the participants.

https://doi.org/10.1371/journal.pone.0188073.g005

Download:

Fig 6. The variance maps of the sum of peak values at 15 Hz and 16 Hz in the PSD among all the participants.

https://doi.org/10.1371/journal.pone.0188073.g006

Offline experimental results of brain response under luminance-based stimulation and motion-based stimulation using inter-modulation frequencies

The Oz site experiences the main neuronal activities in the visual area because it is closest to the striate cortex [23]. Fig 7 shows the frequency spectrums of EEG data at the Oz site while one participant was staring at the luminance-based stimulation and motion-based stimulation respectively. The EEG data were dealt with using time domain averaging first. Several combination frequencies can be clearly observed in both spectrum. As for luminance-based stimulus, the peaks in the spectrum occurred at f₁,f₂,2 × f₁,2 × f₂,f₂±f₁,3 × f₂ ± f₁,3 × f₁ + f₂ where f₁ = 5Hz, f₂ = 8Hz. As for motion-based stimulus, the peaks in the spectrum occurred at where f₁ = 5Hz, f₂ = 8Hz. So both the luminance-based and motion-based stimulation using dual inter-modulation frequencies could induce SSVEP at combination frequencies. But there were some differences between them. The induced frequencies by motion-based stimulation were combined by the motion frequencies , not the motion inversion frequencies. And the peaks occurred at the motion inversion frequencies, not the motion frequencies.

Download:

Fig 7. Frequency spectra of EEG data at the Oz site dealt with using time domain averaging on luminance-based and motion-based stimulation using dual inter-modulation frequencies.

(a) Frequency spectra of EEG data at the Oz site with luminance-based stimulation using dual inter-modulation frequencies. (b) Frequency spectra of EEG data at the Oz site with motion-based stimulation using dual inter-modulation frequencies.

https://doi.org/10.1371/journal.pone.0188073.g007

Experimental results of online recognition accuracy

Fig 8 shows the frequency spectrum of EEG data of a single trial at Oz site with luminance-based and motion-based stimulation using dual inter-modulation frequencies. When the dual frequencies were 16+15 Hz, 16+14 Hz, 16+13 Hz, 16+17 Hz, 15+14 Hz, 15+13 Hz, 15+17 Hz, 14+13 Hz, 14+17 Hz, and 13+17 Hz in proper order, in regard to luminance-based stimulation, the main induced frequencies were (16 Hz 15 Hz), (16 Hz 14 Hz), (16 Hz 13 Hz), (16 Hz 17 Hz), (15 Hz 14 Hz), (15 Hz 13 Hz), (15 Hz 17 Hz), (14 Hz 13 Hz), (14 Hz 17 Hz), and (13 Hz 17 Hz) successively as shown in Fig 8(A). When the dual frequencies were 17.1+15 Hz, 17.1+13.3 Hz, 17.1+12 Hz, 17.1+20 Hz, 15+13.3 Hz, 15+12 Hz, 15+20 Hz, 13.3+12 Hz, 13.3+20 Hz, and 12+20 Hz in proper order, as to motion-based stimulation, the main induced frequencies were 16.05 Hz, 15.4 Hz, 14.5 Hz, 18.5 Hz, 14.1 Hz, 13.5 Hz, 17.5 Hz, 12.6 Hz, 16.6 Hz, 16 Hz successively as shown in Fig 8(B).

Download:

Fig 8. Frequency spectrum of EEG data for a single trial at the Oz site with luminance-based and motion-based stimulation using dual inter-modulation frequencies.

(a) Frequency spectra of EEG data for a single trial at the Oz site with luminance-based stimulation using dual inter-modulation frequencies. (b) Frequency spectra of EEG data for a single trial at the Oz site with motion-based stimulation using dual inter-modulation frequencies.

https://doi.org/10.1371/journal.pone.0188073.g008

In total, as to luminance-based stimulation, the main peaks in the spectrum occurred at f₁ and f₂. The amplitudes at the combination frequencies in the luminance-based stimulation were not outstanding. So Y₁ in forum (5) was chosen as the reference signals to perform CCA. As to motion-based stimulation, the main peaks in the spectrum occurred at . So Y₂ in forum (5) was chosen as the reference signals to perform CCA.

CCA was used to calculate the recognition accuracy of luminance-based stimulation and motion-based stimulation for each subjects. The recognition accuracies of luminance-based stimulation and motion-based stimulation of all the subjects according to stimulation duration are shown in Tables 3 and 4 respectively. Noticeably, the recognition accuracy of S10 reached 98% with only 2 s of stimulation. Regarding the average accuracies, the luminance-based stimulation had higher recognition accuracy than the motion-based stimulation did when the duration was lower than 4 s. And both the recognition accuracies of the two stimulations were decrease sharply as the duration decreased. When the stimulation duration exceeded 4 s, the change of recognition accuracy tended to be stable and the motion-based stimulation had higher recognition accuracy than the luminance-based stimulation did.

Download:

Table 3. Online classification accuracy of luminance-based stimulation according to stimulation duration.

https://doi.org/10.1371/journal.pone.0188073.t003

Download:

Table 4. Online classification accuracy with motion-based stimulation according to stimulation duration.

https://doi.org/10.1371/journal.pone.0188073.t004

Fig 9 shows the mean information transfer rate (ITR) of subjects with different stimulation duration. The ITR reached the highest value at 3 s for 34.7836 bits/min and 39.2856 bits/min with luminance-based and motion-based stimulation respectively. In total, the ITRs of the motion-based stimulation were higher than that of the luminance-based stimulation for the different durations except 1 s.

Download:

Fig 9. Mean of ITR of all the subjects with different stimulation duration.

https://doi.org/10.1371/journal.pone.0188073.g009

Discussion

In this study, luminance-based and motion-based stimulations using inter-modulation frequencies were proposed and these two methods were introduced as new tools for BCI systems to extend related research. Characteristics of EEG response to luminance-based stimulation and motion-based stimulation using multiple inter-modulation frequencies were illustrated. Both offline analysis and online testing demonstrated the feasibility of the proposed methods to encode more targets with limited frequencies.

The inter-modulation frequencies were first reported by Ratliff's group [24]. Regan [20] described the nonlinearity of the EEG response to stimulus using dual inter-modulation frequencies. Several follow-up studies reported similar results. T M Srihari Mukesh [25] proposed a novel dual frequency stimulation method to increase the number of selections in BCI. Only three frequencies generated six selections in their study. And the peaks in the spectrum mainly occurred at F₁,F₂,F₁ + F₂,2 × F₂,F₁ + 2 × F₂, 2 × F₁+ 2 × F₂,3 × F₂,F₁ + 3 × F₂,2 × F₁ + 3 × F₂. Shyu et al. [11] obtained a dual-frequency stimulation with two flickering LEDs. The peaks mainly occurred at F₁,F₂,2 × F₂–F₁,2 × F₁–F₂. Min Hye Chang, etc [26] proposed an amplitude-modulated visual stimulation which combined the advantage of both low and high frequencies with dual frequencies. And the peaks occurred at 2 × f_c,2 × f_m,f_c ± f_m,f_c ± 3 × f_m,2 × f_c ± 4 × f_m. All the studies motioned above were dual frequencies stimulus and all the induced frequencies reflected the nonlinearity of the brain. This study extended dual frequencies to multiple frequencies. The results demonstrated that simultaneous modulation of stimulus luminance with multiple frequencies can induce the corresponding frequencies. And at least five stimulus frequencies could induced simultaneously. What's more, the combination frequencies were not significant. That is to say the nonlinearity of the brain response to our proposed stimulus using multiple inter-modulation frequencies from 12 Hz to 16 Hz was not obvious. What's more, our results showed the brain had a certain positive correlation with luminance even using inter-modulation frequencies. With the increase of one amplitude of the stimulation using inter-modulation frequencies, the peak values at corresponding frequencies in the PSD increased. When the sum of amplitudes at the stimulus frequencies of the stimulus was kept the same, the sum of peak values at corresponding frequencies in the PSD of EEG demonstrated no significant difference.

On the other hand, in Xiaogang Chen's research [22, 27], not only simultaneous modulation of stimulus luminance, but also of stimulus color were considered. F_i ± f_i,2F_i ± f_i were the main inter-modulation frequencies in their study. It was known that luminance and color are all reflected in the P-pathway in primary visual cortex. However, the motion-based stimulation was reflected in the M-pathway. So in order to get a comprehensive understanding of the primary visual cortex response to visual stimulus, motion-based stimulation with equal luminance using inter-modulation frequencies was proposed in this study. The results in this study showed that the brain response to the proposed motion-based stimulation and luminance-based stimulation using inter-modulation frequencies had certain similarities in low frequency bands. Both stimulation methods of them induced combination frequencies. And the results showed the nonlinearity of the brain. But the induced frequencies by motion-based stimulation were combined by the motion frequencies, not the motion inversion frequencies. And there were no peaks that occurred at the motion frequencies. Besides it was still unknown how these frequencies were produced. Zhenghua Wu [28] proposed one theory that different SSVEP neural networks exist whose strongest response are located in different frequency bands by comparing the SSVEP power under a single-eye stimulus and a simultaneous, dual-eye stimulus. A hypothesis was given in this study. The retina received the visual stimulus signal IS. Due to the brain resonances [29], we supposed that the input signal IS was changed into NS when the visual stimulus was transferred to the primary visual cortex. NS contained the abundant inter-modulation frequencies f₁ ± f₂ (i = 2), 2 × f₁ ± f₂,2 × f₂ ± f₁ (i = 3), 3 × f₁ ± f₂,3 × f₂ ± f₁ (i = 4) and so on.

(8)

(9)

Due to different sensitivities of the brain to different frequencies, some combination frequencies did not occur. And the amplitudes of the combination frequencies were different. So induced combination frequencies reported by other studies were different.

Besides, simultaneous modulation of stimulus was used to increase the number of targets using limited frequencies. Actually, n frequencies could totally generate selections in our study. And our stimulation only contained one flickering LED or one target with multiple frequencies which did not cause the 'attention-shift' problem noted in previous studies [10, 11, 30]. What's more, Hakvoort [31] mentioned that more than 16% of the selections were misclassified because of the multiple harmonics using conventional CCA. In this study, Y₁ and Y₂ in formula (5) were chosen as the reference signals for the luminance-based and motion-based stimulations respectively. And online test showed the average ITRs reached 34.7836 bits/min and 39.2856 bits/min respectively. This meant that reference signals using CCA should be chosen based on the stimulus. In regard to ITR, the motion-based stimulation was better than the luminance-based stimulation. But the motion-based stimulation was difficult to extend to multiple inter-modulation frequencies and only had a regular rule with dual inter-modulation frequencies. This was not the case with luminance-based stimulation. For the further research, the classification algorithms, stimulus frequencies, and induced frequencies still need to be optimized to increase the ITR and recognition accuracy.

Conclusion

The present study presented a thorough investigation on stimulation for SSVEP using inter-modulation frequencies. Importantly, luminance-based stimulation using one to five inter-modulation frequencies could induce the corresponding stimulus frequencies in the primary visual cortex. And the response maintained a certain positive correlation with luminance. Additionally, the brain response to motion-based stimulation with equal luminance had similar characteristics with that of luminance-based stimulation. And a hypothesis was presented to explain the reason for the occurrence of combination frequencies. Furthermore, the ITRs in the online test indicated that both of the proposed stimulations were feasible for multi-class SSVEP-BCI systems.

Supporting information

S1 File. The main coding algorithms for producing the stimulation with dual frequencies.

https://doi.org/10.1371/journal.pone.0188073.s001

(M)

S2 File. The peak values of PSD at corresponding stimulus frequencies in E1 task1.

https://doi.org/10.1371/journal.pone.0188073.s002

(SAV)

S3 File. The sum of the peak values of PSD at corresponding stimulus frequencies in E1 task1.

https://doi.org/10.1371/journal.pone.0188073.s003

(SAV)

S4 File. The sum of peak values of PSD at 15 Hz and 16 Hz in E1 task2.

https://doi.org/10.1371/journal.pone.0188073.s004

(SAV)

S5 File. The frequency spectrum of EEG data in E2.

https://doi.org/10.1371/journal.pone.0188073.s005

(XLSX)

S6 File. The frequency spectrum of EEG data in E3.

https://doi.org/10.1371/journal.pone.0188073.s006

(XLSX)

S7 File. The mean of ITR of all the subjects with different stimulation duration.

https://doi.org/10.1371/journal.pone.0188073.s007

(XLSX)

S1 Video. The chessboard stimulation using dual inter-modulation frequencies.

https://doi.org/10.1371/journal.pone.0188073.s008

(MP4)

Acknowledgments

The authors would like to thank the subjects for participating in these experiments and the anonymous reviewers for their helpful comments.

References

1. Vialatte FB, Maurice M, Dauwels J, Cichocki A. Steady-state visually evoked potentials: focus on essential paradigms and future perspectives. Progress in Neurobiology. 2010;90(4):418–38. pmid:19963032
- View Article
- PubMed/NCBI
- Google Scholar
2. Herrmann CS. Human EEG responses to 1–100 Hz flicker: resonance phenomena in visual cortex and their potential correlation to cognitive phenomena. Exp Brain Res. 2001;137(3):346–53.
- View Article
- Google Scholar
3. Wang Y, Wang R, Gao X, Hong B, Gao S. A practical VEP-based brain-computer interface. IEEE Transactions on Neural Systems & Rehabilitation Engineering. 2006;14(2):234–9.
- View Article
- Google Scholar
4. Wang YT, Wang Y, Jung TP. A cell-phone-based brain-computer interface for communication in daily life. J Neural Eng. 2011;8(2):233–40.
- View Article
- Google Scholar
5. Ortner R, Allison BZ, Korisek G, Gaggl H, Pfurtscheller G. An SSVEP BCI to control a hand orthosis for persons with tetraplegia. IEEE Transactions on Neural Systems & Rehabilitation Engineering A Publication of the IEEE Engineering in Medicine & Biology Society. 2011;19(1):1–5.
- View Article
- Google Scholar
6. Müller SM, Diez PF, Bastosfilho TF, Sarcinellifilho M, Mut V, Laciar E, editors. SSVEP-BCI implementation for 37–40 Hz frequency range. International Conference of the IEEE Engineering in Medicine & Biology Society, Embc; 2011.
- View Article
- Google Scholar
7. Volosyak I. SSVEP-based Bremen-BCI interface—boosting information transfer rates. J Neural Eng. 2011;8(3):292–301.
- View Article
- Google Scholar
8. Lee PL, Sie JJ, Liu YJ, Wu CH, Lee MH, Shu CH, et al. An SSVEP-Actuated Brain Computer Interface Using Phase-Tagged Flickering Sequences: A Cursor System. Ann Biomed Eng. 2010;38(7):2383–97. pmid:20177780
- View Article
- PubMed/NCBI
- Google Scholar
9. Zhang F, Han C, Li L, Zhang X, Xie J, Li Y. Research on High-Frequency Combination Coding-Based SSVEP-BCIs and Its Signal Processing Algorithms. Shock & Vibration. 2015;2015(12):1–12.
- View Article
- Google Scholar
10. Teng F, Choong AM, Gustafson S, Waddell D, Lawhead P, Chen Y, editors. Steady state visual evoked potentials by dual sine waves. Southeast Regional Conference, 2010, Oxford, Ms, Usa, April; 2010.
11. Shyu KK, Lee PL, Liu YJ, Sie JJ. Dual-frequency steady-state visual evoked potential for brain computer interface. Neurosci Lett. 2010;483(483):28–31.
- View Article
- Google Scholar
12. Min HC, Lee JS, Heo J, Park KS. Eliciting dual-frequency SSVEP using a hybrid SSVEP-P300 BCI. J Neurosci Meth. 2016;258:104.
- View Article
- Google Scholar
13. Mishkin M, Ungerleider LG. Contribution of striate inputs to the visuospatial functions of parieto-preoccipital cortex in monkeys. Behav Brain Res. 1982;6(1):57–77. pmid:7126325
- View Article
- PubMed/NCBI
- Google Scholar
14. Delon-Martin C, Gobbelé R, Buchner H, Haug BA, Antal A, Darvas F, et al. Temporal pattern of source activities evoked by different types of motion onset stimuli. Neuroimage. 2006;31(4):1567. pmid:16580846
- View Article
- PubMed/NCBI
- Google Scholar
15. Ales JM, Norcia AM. Assessing direction-specific adaptation using the steady-state visual evoked potential: results from EEG source imaging. J Vision. 2009;9(7):8.
- View Article
- Google Scholar
16. Chien YY, Lin FC, Zao JK, Chou CC, Huang YP, Kuo HY, et al. Polychromatic SSVEP stimuli with subtle flickering adapted to brain-display interactions. J Neural Eng. 2017;14(1):016018. pmid:28000607
- View Article
- PubMed/NCBI
- Google Scholar
17. Xie J, Xu G, Jing W, Feng Z, Zhang Y. Steady-State Motion Visual Evoked Potentials Produced by Oscillating Newton's Rings: Implications for Brain-Computer Interfaces. Plos One. 2011;7(6):e39707.
- View Article
- Google Scholar
18. Z L, C Z, W W, X G. Frequency recognition based on canonical correlation analysis for SSVEP-based BCIs. Ieee T Bio-Med Eng. 2006;54(6 Pt 2):1172–6.
- View Article
- Google Scholar
19. Chen X, Wang Y, Nakanishi M, Gao X, Jung TP, Gao S. High-speed spelling with a noninvasive brain-computer interface. P Natl Acad Sci USA. 2015;112(44):6058–67.
- View Article
- Google Scholar
20. Bromm B. Human brain electrophysiology. Evoked potentials and evoked magnetic fields in science and medicine: D. Regan, Elsevier, New York, 1989, 672 pp., 372 figs., approx. 2800 references, price: U.S. $140.00, ISBN 0-444-01324-5. Pain. 1989;39(3):371–2.
- View Article
- Google Scholar
21. Giani AS, Ortiz E, Belardinelli P, Kleiner M, Preissl H, Noppeney U. Steady-state responses in MEG demonstrate information integration within but not across the auditory and visual senses. Neuroimage. 2012;60(2):1478–89. pmid:22305992
- View Article
- PubMed/NCBI
- Google Scholar
22. Chen X, Chen Z, Gao S, Gao X. Brain-computer interface based on intermodulation frequency. J Neural Eng. 2013;10(6):066009. pmid:24140740
- View Article
- PubMed/NCBI
- Google Scholar
23. Ng KB, Bradley AP, Cunnington R. Stimulus specificity of a steady-state visual-evoked potential-based brain-computer interface. J Neural Eng. 2012;9(3):036008. pmid:22589242
- View Article
- PubMed/NCBI
- Google Scholar
24. Zemon V, Ratliff F. Intermodulation components of the visual evoked potential: Responses to lateral and superimposed stimuli. Biol Cybern. 1984;50(6):401–8. pmid:6487677
- View Article
- PubMed/NCBI
- Google Scholar
25. Srihari Mukesh TM, Jaganathan V, Reddy MR. A novel multiple frequency stimulation method for steady state VEP based brain computer interfaces. Physiol Meas. 2006;27(1):61–71. pmid:16365511
- View Article
- PubMed/NCBI
- Google Scholar
26. Chang MH, Baek HJ, Lee SM, Park KS. An amplitude-modulated visual stimulation for reducing eye fatigue in SSVEP-based brain-computer interfaces. Clinical Neurophysiology Official Journal of the International Federation of Clinical Neurophysiology. 2014;125(7):1380–91. pmid:24368034
- View Article
- PubMed/NCBI
- Google Scholar
27. Xiaogang C, Yijun W, Shangen Z, Shangkai G, Yong H, Xiaorong G. A novel stimulation method for multi-class SSVEP-BCI using intermodulation frequencies. J Neural Eng. 2017;14(2):026013. pmid:28091397
- View Article
- PubMed/NCBI
- Google Scholar
28. Wu Z. Physical connections between different SSVEP neural networks. Sci Rep-Uk. 2016;6:22801.
- View Article
- Google Scholar
29. Belmonte M. Shifts of visual spatial attention modulate a steady-state visual evoked potential. Brain Research. Cognitive Brain Research. 1998;6(4):295–307. pmid:9593953
- View Article
- PubMed/NCBI
- Google Scholar
30. Yan Z, Gao X, Bin G, Hong B, Gao S, editors. A half-field stimulation pattern for SSVEP-based brain-computer interface. Conference: International Conference of the IEEE Engineering in Medicine & Biology Society IEEE Engineering in Medicine & Biology Society Conference; 2009.
31. Hakvoort G, Reuderink B, Obbink M. Comparison of PSDA and CCA detection methods in a SSVEP-based BCI-system. Centre for Telematics & Information Technology University of Twente. 2011.

[ref1] 1. Vialatte FB, Maurice M, Dauwels J, Cichocki A. Steady-state visually evoked potentials: focus on essential paradigms and future perspectives. Progress in Neurobiology. 2010;90(4):418–38. pmid:19963032
View Article
PubMed/NCBI
Google Scholar

[2] View Article

[3] PubMed/NCBI

[4] Google Scholar

[ref2] 2. Herrmann CS. Human EEG responses to 1–100 Hz flicker: resonance phenomena in visual cortex and their potential correlation to cognitive phenomena. Exp Brain Res. 2001;137(3):346–53.
View Article
Google Scholar

[6] View Article

[7] Google Scholar

[ref3] 3. Wang Y, Wang R, Gao X, Hong B, Gao S. A practical VEP-based brain-computer interface. IEEE Transactions on Neural Systems & Rehabilitation Engineering. 2006;14(2):234–9.
View Article
Google Scholar

[9] View Article

[10] Google Scholar

[ref4] 4. Wang YT, Wang Y, Jung TP. A cell-phone-based brain-computer interface for communication in daily life. J Neural Eng. 2011;8(2):233–40.
View Article
Google Scholar

[12] View Article

[13] Google Scholar

[ref5] 5. Ortner R, Allison BZ, Korisek G, Gaggl H, Pfurtscheller G. An SSVEP BCI to control a hand orthosis for persons with tetraplegia. IEEE Transactions on Neural Systems & Rehabilitation Engineering A Publication of the IEEE Engineering in Medicine & Biology Society. 2011;19(1):1–5.
View Article
Google Scholar

[15] View Article

[16] Google Scholar

[ref6] 6. Müller SM, Diez PF, Bastosfilho TF, Sarcinellifilho M, Mut V, Laciar E, editors. SSVEP-BCI implementation for 37–40 Hz frequency range. International Conference of the IEEE Engineering in Medicine & Biology Society, Embc; 2011.
View Article
Google Scholar

[18] View Article

[19] Google Scholar

[ref7] 7. Volosyak I. SSVEP-based Bremen-BCI interface—boosting information transfer rates. J Neural Eng. 2011;8(3):292–301.
View Article
Google Scholar

[21] View Article

[22] Google Scholar

[ref8] 8. Lee PL, Sie JJ, Liu YJ, Wu CH, Lee MH, Shu CH, et al. An SSVEP-Actuated Brain Computer Interface Using Phase-Tagged Flickering Sequences: A Cursor System. Ann Biomed Eng. 2010;38(7):2383–97. pmid:20177780
View Article
PubMed/NCBI
Google Scholar

[24] View Article

[25] PubMed/NCBI

[26] Google Scholar

[ref9] 9. Zhang F, Han C, Li L, Zhang X, Xie J, Li Y. Research on High-Frequency Combination Coding-Based SSVEP-BCIs and Its Signal Processing Algorithms. Shock & Vibration. 2015;2015(12):1–12.
View Article
Google Scholar

[28] View Article

[29] Google Scholar

[ref10] 10. Teng F, Choong AM, Gustafson S, Waddell D, Lawhead P, Chen Y, editors. Steady state visual evoked potentials by dual sine waves. Southeast Regional Conference, 2010, Oxford, Ms, Usa, April; 2010.

[ref11] 11. Shyu KK, Lee PL, Liu YJ, Sie JJ. Dual-frequency steady-state visual evoked potential for brain computer interface. Neurosci Lett. 2010;483(483):28–31.
View Article
Google Scholar

[32] View Article

[33] Google Scholar

[ref12] 12. Min HC, Lee JS, Heo J, Park KS. Eliciting dual-frequency SSVEP using a hybrid SSVEP-P300 BCI. J Neurosci Meth. 2016;258:104.
View Article
Google Scholar

[35] View Article

[36] Google Scholar

[ref13] 13. Mishkin M, Ungerleider LG. Contribution of striate inputs to the visuospatial functions of parieto-preoccipital cortex in monkeys. Behav Brain Res. 1982;6(1):57–77. pmid:7126325
View Article
PubMed/NCBI
Google Scholar

[38] View Article

[39] PubMed/NCBI

[40] Google Scholar

[ref14] 14. Delon-Martin C, Gobbelé R, Buchner H, Haug BA, Antal A, Darvas F, et al. Temporal pattern of source activities evoked by different types of motion onset stimuli. Neuroimage. 2006;31(4):1567. pmid:16580846
View Article
PubMed/NCBI
Google Scholar

[42] View Article

[43] PubMed/NCBI

[44] Google Scholar

[ref15] 15. Ales JM, Norcia AM. Assessing direction-specific adaptation using the steady-state visual evoked potential: results from EEG source imaging. J Vision. 2009;9(7):8.
View Article
Google Scholar

[46] View Article

[47] Google Scholar

[ref16] 16. Chien YY, Lin FC, Zao JK, Chou CC, Huang YP, Kuo HY, et al. Polychromatic SSVEP stimuli with subtle flickering adapted to brain-display interactions. J Neural Eng. 2017;14(1):016018. pmid:28000607
View Article
PubMed/NCBI
Google Scholar

[49] View Article

[50] PubMed/NCBI

[51] Google Scholar

[ref17] 17. Xie J, Xu G, Jing W, Feng Z, Zhang Y. Steady-State Motion Visual Evoked Potentials Produced by Oscillating Newton's Rings: Implications for Brain-Computer Interfaces. Plos One. 2011;7(6):e39707.
View Article
Google Scholar

[53] View Article

[54] Google Scholar

[ref18] 18. Z L, C Z, W W, X G. Frequency recognition based on canonical correlation analysis for SSVEP-based BCIs. Ieee T Bio-Med Eng. 2006;54(6 Pt 2):1172–6.
View Article
Google Scholar

[56] View Article

[57] Google Scholar

[ref19] 19. Chen X, Wang Y, Nakanishi M, Gao X, Jung TP, Gao S. High-speed spelling with a noninvasive brain-computer interface. P Natl Acad Sci USA. 2015;112(44):6058–67.
View Article
Google Scholar

[59] View Article

[60] Google Scholar

[ref20] 20. Bromm B. Human brain electrophysiology. Evoked potentials and evoked magnetic fields in science and medicine: D. Regan, Elsevier, New York, 1989, 672 pp., 372 figs., approx. 2800 references, price: U.S. $140.00, ISBN 0-444-01324-5. Pain. 1989;39(3):371–2.
View Article
Google Scholar

[62] View Article

[63] Google Scholar

[ref21] 21. Giani AS, Ortiz E, Belardinelli P, Kleiner M, Preissl H, Noppeney U. Steady-state responses in MEG demonstrate information integration within but not across the auditory and visual senses. Neuroimage. 2012;60(2):1478–89. pmid:22305992
View Article
PubMed/NCBI
Google Scholar

[65] View Article

[66] PubMed/NCBI

[67] Google Scholar

[ref22] 22. Chen X, Chen Z, Gao S, Gao X. Brain-computer interface based on intermodulation frequency. J Neural Eng. 2013;10(6):066009. pmid:24140740
View Article
PubMed/NCBI
Google Scholar

[69] View Article

[70] PubMed/NCBI

[71] Google Scholar

[ref23] 23. Ng KB, Bradley AP, Cunnington R. Stimulus specificity of a steady-state visual-evoked potential-based brain-computer interface. J Neural Eng. 2012;9(3):036008. pmid:22589242
View Article
PubMed/NCBI
Google Scholar

[73] View Article

[74] PubMed/NCBI

[75] Google Scholar

[ref24] 24. Zemon V, Ratliff F. Intermodulation components of the visual evoked potential: Responses to lateral and superimposed stimuli. Biol Cybern. 1984;50(6):401–8. pmid:6487677
View Article
PubMed/NCBI
Google Scholar

[77] View Article

[78] PubMed/NCBI

[79] Google Scholar

[ref25] 25. Srihari Mukesh TM, Jaganathan V, Reddy MR. A novel multiple frequency stimulation method for steady state VEP based brain computer interfaces. Physiol Meas. 2006;27(1):61–71. pmid:16365511
View Article
PubMed/NCBI
Google Scholar

[81] View Article

[82] PubMed/NCBI

[83] Google Scholar

[ref26] 26. Chang MH, Baek HJ, Lee SM, Park KS. An amplitude-modulated visual stimulation for reducing eye fatigue in SSVEP-based brain-computer interfaces. Clinical Neurophysiology Official Journal of the International Federation of Clinical Neurophysiology. 2014;125(7):1380–91. pmid:24368034
View Article
PubMed/NCBI
Google Scholar

[85] View Article

[86] PubMed/NCBI

[87] Google Scholar

[ref27] 27. Xiaogang C, Yijun W, Shangen Z, Shangkai G, Yong H, Xiaorong G. A novel stimulation method for multi-class SSVEP-BCI using intermodulation frequencies. J Neural Eng. 2017;14(2):026013. pmid:28091397
View Article
PubMed/NCBI
Google Scholar

[89] View Article

[90] PubMed/NCBI

[91] Google Scholar

[ref28] 28. Wu Z. Physical connections between different SSVEP neural networks. Sci Rep-Uk. 2016;6:22801.
View Article
Google Scholar

[93] View Article

[94] Google Scholar

[ref29] 29. Belmonte M. Shifts of visual spatial attention modulate a steady-state visual evoked potential. Brain Research. Cognitive Brain Research. 1998;6(4):295–307. pmid:9593953
View Article
PubMed/NCBI
Google Scholar

[96] View Article

[97] PubMed/NCBI

[98] Google Scholar

[ref30] 30. Yan Z, Gao X, Bin G, Hong B, Gao S, editors. A half-field stimulation pattern for SSVEP-based brain-computer interface. Conference: International Conference of the IEEE Engineering in Medicine & Biology Society IEEE Engineering in Medicine & Biology Society Conference; 2009.

[ref31] 31. Hakvoort G, Reuderink B, Obbink M. Comparison of PSDA and CCA detection methods in a SSVEP-based BCI-system. Centre for Telematics & Information Technology University of Twente. 2011.

Figures

Abstract

Introduction

Materials and methods

Ethics statement

Design of luminance-based stimulation

Design of motion-based stimulation

Experimental design

E1: Offline experiment of brain response under luminance-based stimulation using inter-modulation frequencies.

E2: Offline experiment of comparisons between luminance-based and motion-based stimulation using dual inter-modulation frequencies.

E3: Online experiment of luminance-based and motion-based stimulation using inter-modulation frequencies.

Data analysis

Canonical correlation analysis.

CCA-based spatial filter.

Power spectrum density.

Statistical analysis.

Results

Offline experimental results of brain response under luminance-based stimulation using inter-modulation frequencies

Offline experimental results of brain response under luminance-based stimulation and motion-based stimulation using inter-modulation frequencies

Experimental results of online recognition accuracy

Discussion

Conclusion

Supporting information

S1 File. The main coding algorithms for producing the stimulation with dual frequencies.

S2 File. The peak values of PSD at corresponding stimulus frequencies in E1 task1.

S3 File. The sum of the peak values of PSD at corresponding stimulus frequencies in E1 task1.

S4 File. The sum of peak values of PSD at 15 Hz and 16 Hz in E1 task2.

S5 File. The frequency spectrum of EEG data in E2.

S6 File. The frequency spectrum of EEG data in E3.

S7 File. The mean of ITR of all the subjects with different stimulation duration.

S1 Video. The chessboard stimulation using dual inter-modulation frequencies.

Acknowledgments

References