Information

Difference between SSVEP and P300

Difference between SSVEP and P300



We are searching data for your request:

Forums and discussions:
Manuals and reference books:
Data from registers:
Wait the end of the search in all databases.
Upon completion, a link will appear to access the found materials.

I have read about steady state visually evoked potentials (SSVEP) and P300 as different subjects. But it seems that they are related to each other.

Is P300 a kind of SSVEP?


Not at all. The SSVEP is a stimulus-driven reaction, a fluctuation in the neural signal driven by the fluctuation in the observed visual signal. The SSVEP begins and ends with the presence of the visual signal.

The P300 a reflection of cognitive processing, an increased response in the presence of deviance detection, decision making, attention, categorization… some higher order processing. The sensory stimulation can be long gone from the display, but as long as it's still being processed by the cognitive apparatus, the P300 can arise. The 300 stands for milliseconds, i.e. it roughly begins about that much after stimulus onset. The only connection between the two is that they're electrophysiological measures, i.e. you can capture both with M/EEG, and that you might observe both in the same brain regions.


@Ana is basically correct (possibly baring the fact that the P3 is famous for being latency variant, not stable at 300msec), and I really don't want to contradict her or steal her checkmark, but since it's recent and interesting, I would like to point to some recent research by O'Connell and colleagues looking at correlations and similarities between the two measures, as the primary EEG indexes of attention and evidence accumulation. Incidentally, it also shows the differences in topography, polarity and size for the two, as well as the slight dissociation in time course. (Note also that the P3 is a distinct, monophasic peak - it has an amplitude; the SSVEP is a periodic pattern, strictly speaking, it rather has power.)

In the following picture, the close correspondence between the SSVEP and the P3 is demonstrated; see the figure caption below.

(a) Target contrast reduction time course. (b) Grand-average reaction time (RT) distribution (19 subjects). Single trials were sorted by reaction time and divided into three equal-sized bins. (c) Neural signals undergoing gradual changes on the timescale of the physical contrast change, aligned to stimulus onset (left) and response (right). Vertical dashed lines denote mean reaction time. Markers running along the bottom of each plot indicate time points at which a linear regression of signal amplitude onto reaction time reached significance (P < 0.05). (d) The cumulative sum of SSVEP divergence from baseline provided a better fit for the dynamics of the LHB and CPP signals than the raw SSVEP amplitude. To facilitate visualization of the comparison, we baseline-subtracted and inverted SSVEP amplitude. LHB was transformed for each participant so that it reflected the percentage decrease from baseline, then inverted. (e) Single-trial surface plots showing the temporal relationship between each neural signal (normalized relative to each individual's baseline average) and target detection latency (curved black line). Single-trial SSVEP, LHB and CPP signals were pooled across participants, sorted by reaction time and smoothed over bins of 50 trials with a Gaussian-weighted moving average using the EEGLAB toolbox38. The latency of action execution was closely tied to that of the two decision signals, but was less closely related to the sensory evidence signal. (f) Signal scalp topographies, all with pretarget baseline subtracted. Color bars in e and f represent amplitudes.


Visual SSVEP¶

The steady-state visual evoked potential (SSVEP) is a repetitive evoked potential that is naturally produced when viewing stimuli flashing between a range of 6-75hz. Electrical activity at the same frequency as the visual stimulation can be detected in the occipital areas of the brain, likely due to the perceptual recreation of the stimulus in the primary visual cortex.

The SSVEP is often used in BCI applications due to its ease of detection and the amount of information that a user can communicate due to the high potential frequency resolution of the SSVEP.

In this notebook, we will use the Muse EEG headband with an extra occipital electrode to detect the SSVEP and evaluate it’s use in SSVEP-based BCIs.

Although the SSVEP is detectable at the default temporal electrodes, it can be seen much more clearly directly over the occipital cortex.

The Muse 2016 supports the addition of an extra electrode which can be connected through the devices microUSB charging port.

Instructions on how to build an extra electrode for Muse Working with the extra electrode For this experiment, the extra electrode should be placed at POz, right at the back of the skull. It can be secured in place with a bandana or a hat


[Relationship between P300 and intelligence quotient in severe head injury patients]

Objective: To explore the change of brain cognition function in severe head injury and the correlation between P300 in event-related potential (ERP) and intelligence quotient (IQ).

Methods: Auditory P300 was measured and intelligence quotient was tested by Wechsler Adult Intelligence Scale Revised in China (WAIS-RC) in 40 severe head injury patients. Auditory P300 was measured in 40 normal healthy people as control group.

Results: The latencies of P300 in severe head injury patients group were longer than in control group. There was a significant difference between the two groups (P< 0.01). There were significant negative correlations between P300 latency and VIQ and FIQ respectively (r=-0.335,-0.344, P<0.05).

Conclusion: ERP might be taken as an objective index for measuring the change of the brain cognition function in patients with severe head injury.


Contents

Early observations of the P300 (more specifically, the component that would later be named the P3b) were reported in the mid-1960s. In 1964, researchers Chapman and Bragdon [4] found that ERP responses to visual stimuli differed depending on whether the stimuli had meaning or not. They showed subjects two kinds of visual stimuli: numbers and flashes of light. Subjects viewed these stimuli one at a time in a sequence. For every two numbers, the subjects were required to make simple decisions, such as telling which of the two numbers was numerically smaller or larger, which came first or second in the sequence, or whether they were equal. When examining evoked potentials to these stimuli (i.e., ERPs), Chapman and Bragdon found that both the numbers and the flashes elicited the expected sensory responses (e.g., visual N1 components), and that the amplitude of these responses varied in an expected fashion with the intensity of the stimuli. They also found that the ERP responses to the numbers, but not to the light flashes, contained a large positivity that peaked around 300 ms after the stimulus appeared. Chapman and Bragdon speculated that this differential response to the numbers, which came to be known as the P300 response, resulted from the fact that the numbers were meaningful to the participants, based on the task that they were asked to perform.

In 1965, Sutton and colleagues published results from two experiments that further explored this late positivity. They presented subjects with either a cue that indicated whether the following stimulus would be a click or a flash, or a cue which required subjects to guess whether the following stimulus would be a click or a flash. They found that when subjects were required to guess what the following stimulus would be, the amplitude of the "late positive complex" [5] was larger than when they knew what the stimulus would be. In a second experiment, they presented two cue types. For one cue there was a 2 in 3 chance that the following stimulus would be a click and a 1 in 3 chance that the following stimulus would be a flash. The second cue type had probabilities that were the reverse of the first. They found that the amplitude of the positive complex was larger in response to the less probable stimuli, or the one that only had a 1 in 3 chance of appearing. Another important finding from these studies is that this late positive complex was observed for both the clicks and flashes, indicating that the physical type of the stimulus (auditory or visual) did not matter.

In later studies published in 1967, Sutton and colleagues had subjects guess whether they would hear one click or two clicks. [6] They again observed a positivity around 300 ms after the second click occurred – or would have occurred, in the case of the single click. They also had subjects guess how long the interval between clicks might be, and in this case, the late positivity occurred 300 ms after the second click. This shows two important findings: first, that this late positivity occurred when uncertainty about the type of click was resolved, and second, that even an absence of a stimulus would elicit the late positive complex, if said stimulus was relevant to the task. These early studies encouraged the use of ERP methods to study cognition and provided a foundation for the extensive work on the P300 in the decades that followed.

The P3a, or novelty P3, [7] has a positive-going amplitude that displays maximum amplitude over frontal/central electrode sites and has a peak latency in the range of 250–280 ms. The P3a has been associated with brain activity related to the engagement of attention (especially the orienting, involuntary shifts to changes in the environment), and the processing of novelty. [8]

The P3b has a positive-going amplitude (usually relative to a reference behind the ear or the average of two such references) that peaks at around 300 ms, and the peak will vary in latency from 250 to 500 ms or more, depending upon the task and the individual subject response. [3] Amplitudes are typically highest on the scalp over parietal brain areas. [3] The P3b has been a prominent tool used to study cognitive processes, especially psychology research on information processing. Generally speaking, improbable events will elicit a P3b, and the less probable the event, the larger the P3b amplitude. [9] This was shown to be true both for the overall probability and for the local probability. [2] However, in order to elicit a P3b, the improbable event must be related to the task at hand in some way (for example, the improbable event could be an infrequent target letter in a stream of letters, to which a subject might respond with a button press). The P3b can also be used to measure how demanding a task is on cognitive workload. [9]

Since the initial discovery of the P300, research has shown that the P300 has two subcomponents. The subcomponents are the novelty P3, or P3a, and the classic P300, which has since been renamed P3b. [10]

Since the mid-1980s, one of the most discussed uses of ERPs such as the P300 is related to lie detection. In a proposed "guilty knowledge test" [11] a subject is interrogated via the oddball paradigm much as they would be in a typical lie-detector situation. This practice has recently enjoyed increased legal permissibility [ citation needed ] while conventional polygraphy has seen its use diminish, in part owing to the unconscious and uncontrollable aspects of the P300. The technique relies on reproducible elicitation of the P300 wave, central to the idea of a Memory and Encoding Related Multifaceted Electroencephalographic Response (MERMER) developed by Dr. Lawrence Farwell.

Applications in brain-computer interfacing (BCI) have also been proposed. [12] [13] [14] The P300 has a number of desirable qualities that aid in implementation of such systems. First, the waveform is consistently detectable and is elicited in response to precise stimuli. The P300 waveform can also be evoked in nearly all subjects with little variation in measurement techniques, which may help simplify interface designs and permit greater usability. The speed at which an interface is able to operate depends on how detectable the signal is despite "noise." One negative characteristic of the P300 is that the amplitude of the waveform requires averaging of multiple recordings to isolate the signal. This and other post-recording processing steps determine the overall speed of an interface. [13] The algorithm proposed by Farwell and Donchin [15] provides an example of a simple BCI that relies on the unconscious decision making processes of the P300 to drive a computer. A 6×6 grid of characters is presented to the subject, and various columns or rows are highlighted. When a column or row contains the character a subject desires to communicate, the P300 response is elicited (since this character is "special" it is the target stimulus described in the typical oddball paradigm). The combination of the row and column which evoked the response locates the desired character. A number of such trials must be averaged to clear noise from the EEG. The speed of the highlighting determines the number of characters processed per minute. Results from studies using this setup show that normal subjects could achieve a 95% success rate at 3.4–4.3 chars/min. Such success rates are not limited to non-disabled users a study conducted in 2000 revealed that 4 paralyzed participants (one with complete paraplegia, three with incomplete paraplegia) performed as successfully as 10 normal participants. [13]

Scientific research often relies on measurement of the P300 to examine event related potentials, especially with regard to decision making. Because cognitive impairment is often correlated with modifications in the P300, the waveform can be used as a measure for the efficacy of various treatments on cognitive function. Some have suggested its use as a clinical marker for precisely these reasons. There is a broad range of uses for the P300 in clinical research. [16]


METHODS

Design of the study

The most popular and well-researched BCIs are based upon P300, steady-state visual evoked potentials (SSVEP) or motor imagery (MI). Responses can be evoked by stimuli delivered in visual, auditory or tactile modalities. Currently, the most effective BCIs are visual P300 and SSVEP, but these paradigms require complete visual attention, impeding the possibility of using vision at the same time. MI is free from these constraints, but its application is far more complicated than the former two it usually requires extensive prior training for each subject and the achieved information transfer rates (ITRs) are lower than are achievable in the other paradigms (Nicolas-Alonso and Gomez-Gil, 2012). BCIs that do not involve vision can also be based on auditory or tactile stimuli.

We tested four BCIs. The first two were based on the most popular paradigms – visual P300 and SSVEP. As for SSVEP, we used only high frequencies, due to the possibility of inducing photoepileptic attack and annoyance related to low frequency flickers (Fisher et al., 2005). To compensate for a possible dependence on the stimulus modality, we also tested P300-BCI with auditory and vibrotactile stimuli. To facilitate the task and shorten the calibration time we chose the most basic binary BCI setups (yes/no). To stay as close as possible to the “real world” scenario, we used a compact and comfortable EEG headset with a built-in wireless amplifier and water-based electrodes and also included in the testing one representative of an important target group – deeply paralysed persons, who cannot communicate via traditional channels or assistive technologies. This group includes patients after severe brain injury, who awake from coma and evolve into DoC. One of the hallmarks of emergence from DoC is a functional “yes/no” communication. However, due to the deep motor impairments and problems with regaining language functions in these patients, these hallmarks may elude traditional behavioral assessments like (Giacino et al., 2004) – hence the crucial role of BCIs in DoC (Dovgialo et al., 2019).

Hardware and software

EEG was recorded using an 8-channel wireless EEG headset by BrainTech Ltd. (Fig. 1), with water-based electrodes in positions corresponding to C3, Cz, C4, P3, Pz, P4, O1 and O2 from the 10–20 international system, with reference at M1 and ground at M2.

Fig. 1.

Left: prototype of the BrainTech EEG headset with integrated wireless amplifier and water-based electrodes, used in this study. Right: electrode positions from the 10–20 system.

Electrode impedance was monitored online at 125 Hz and kept below 20 kΩ through the duration of procedures. The signal sampling rate was 500 Hz. The design and execution of experiments were based upon the BrainTech BCI framework. Electrode impedances, current potentials and partial classification results were monitored online.

The experiment

Four BCI paradigms were used: Visual P300, Auditory P300, Vibrotactile P300, SSVEP in high frequencies.

Thirty healthy volunteers: 20 females (mean age 24) and 10 males (mean age 22) tested each of these paradigms. The order of BCI sessions was randomly chosen for each subject. Experiments were randomly split into two sessions for each volunteer. Sessions were performed at different days to avoid fatigue of the participants. Each session consisted of calibration and communication. Additionally, one patient (14 years old) from a model hospital for children with severe brain damage, The Alarm Clock Clinic (http://www.klinikabudzik.pl/en), tested the P300-BCIs. This patient, who was in a coma after brain injury, was diagnosed at the time of the experiment with emergence from the minimally conscious state (eMCS) (Giacino et al., 2002) by means of the Coma Recovery Scale-Revised (Giacino et al., 2004), and could communicate via head movements.

The study was approved by the Rectors Committee for Ethics of Research with Human Participants at the University of Warsaw, and informed consent was obtained from the volunteers’ and patients’ legal representatives.

Stimuli in binary P300-BCI

The first three paradigms from the previous section are based upon the same kind of expected response to stimuli, delivered in different modalities – the P300 evoked potential.

In the classical P300 speller, dozens of options (letters or groups of letters) are highlighted in random sequences, and the participant is instructed to count the blinks of the letter of interest. Such conditions conform to the oddball paradigm, where presentations of sequences of repetitive stimuli are infrequently interrupted by a deviant stimulus. Infrequent occurrences of the letter of interest are the deviant stimuli, while all the other blinks are treated as the repetitive stimuli, which the subject ignores. However, in a binary “yes or no” communication there are only two stimuli to choose from, which does not produce the oddball conditions necessary to elicit P300. Therefore, we implemented a scheme (called “MMN presentation paradigm” (Jin et al., 2015)) where the two infrequent stimuli – targets corresponding to the possible choices – are embedded in a stream of a frequently repeated stimulus – a distractor of the third type, which the participant ignores. Targets occur with a probability of 1/6.

Stimuli for visual P300

The words “yes” and “no” were written (in Polish) in black on a white rectangle. Changes of the text color from black to red correspond to the frequent stimulus (distractor), while the actual choices (the target stimulus) were made by counting the appearances of an image of the face of Albert Einstein (Kaufmann et al., 2013) on top of the desired option. The duration of these stimuli was set to 100 ms, with the inter-stimulus interval (ISI) randomized between 200 and 350 ms.

Stimuli in the auditory P300 paradigm

Three different tones: C4 (261.63 Hz), E4 (329.63 Hz) and G4 (392 Hz) played on different instruments were combined with sound spatialization (left, center, right). Left – the lowest tone (synthesized electric piano/synth) – was the rare stimuli (target) for the “yes” selection, center – middle tone (acoustic piano) – was the frequent stimuli (distractor) and right – the highest tone (harpsichord) – was the rare stimulus (target) for “no”. Amplitudes were normalized to perceived equality of the loudness levels. ISI was randomized between 400 and 700 ms, with sound duration 200 ms. Stimuli were presented using plug-in Bose Soundtrue earphones at a level comfortable for the subject. Additionally, the words “yes” and “no” were statically displayed on the screen as a hint for the lateralization during sessions.

Vibrotactile stimuli

Vibrotactile stimulators were taped to left and right hands and to the neck of the subject. The left-hand tactor served as the rare stimulus (target) for “yes”, neck stimuli served as the frequent neutral stimulus (distractor) and the rare stimulus (target) for “no” was on the right hand, as in (Brouwer and Van Erp, 2010). During sessions the words “yes” and “no” were also displayed on the screen. Stimulation duration was 200 ms with ISI randomized between 300 and 450 ms.

Stimuli in SSVEP-BCI

Steady state visually evoked potentials (SSVEPs) can be recorded in response to visual stimulation with specific frequency, within the commonly defined frequency ranges: low (below 12 Hz), medium (12 Hz-30 Hz) and high (above 30 Hz) (Herrmann, 2001). Flicker at low or medium frequencies can be annoying and can trigger an epileptic seizure in vulnerable subjects (Fisher et al., 2005). Stable generation of high frequency stimuli cannot be achieved –for an arbitrary frequency – on a computer screen, and requires hardware control by dedicated electronics (Durka et al., 2012). To conserve the flexibility of rendering messages in the flickering fields, we used the Blinker device from BrainTech Ltd. (https://braintech.pl/blinker/?lang=en). It consists of 320 highlightable fields, with flickering frequency of each field controlled separately by dedicated electronics. To highlight the “yes” and “no” rectangles, the Blinker was configured to create two 6.5 × 4.5 cm flickering fields, observed from about 100 cm.

Calibration

Calibration was performed in short blocks. Sound and brief visual instructions were presented before every block. During the calibration sessions, participants were asked to concentrate on: the face appearing over one of the words (yes/no) in the visual P300, sound corresponding to the yes/no selection in the auditory P300, vibrotactile stimulation corresponding to the yes/no selection in the vibrotactile P300, and the field displaying “yes”, “no”, or the non-blinking field with instruction in SSVEP-BCI.

P300 calibration

After the instruction, the stimulation cycle was started. It consisted of 3–4 target repeats and a pause. Calibration was stopped when 40 target epochs were collected.

SSVEP calibration

The calibration procedure contains blocks of three instructions, asking participants to concentrate her/his attention on the word “yes”, word “no”, and the instruction text. The order was randomized. Before instructions, a 2 second pause allowed participants to rest after the previous stimulation. Texts of the instructions were shown on the top status field and played back as a voice-over. After each instruction, stimulation was started for 6 seconds: the fields with words “yes” and “no” started flickering with different frequencies. After every 2 blocks, the system validated the performance of the current frequency set and, if needed, selected a new set of frequencies to be calibrated. Focusing the participant’s attention on the instruction field allowed the system to collect statistics for the non-control state, when the stimulation was on but the participant was not focusing on any of the control words this allowed for detection of non-control state. Calibration continued until the system found a set of frequencies which provided AUC≥0.8. AUC stands for the area under the ROC curve, used for the assessment of detection as in Dovgialo et al. (2019).

Communication

The communication session started directly after each calibration and consisted of answering 20 simple yes/no questions via BCI (e.g. “Is grass blue?”). It was assumed that the respondents knew the right answers. The proportion of expected yes/no responses was 1:1 and the order of questions was randomized for each session. Each session starts with text and audio instructions explaining the task. The subject was asked to listen to the question and focus on the target stimuli corresponding to the correct answer. After one second, the cyclic presentation of stimuli started and continued until the classifier returned the selected word. When a word was selected, the stimulation cycle stopped and visual feedback was presented: the background of the selected word was highlighted red (if the answer was incorrect) or green (if answer was correct).

Data analysis

Classification of visual P300 and SSVEP potentials relied on derivations O1, O2, Pz and Cz. Auditory and vibrotactile paradigms used C3, C4, Cz and Pz.

Raw signal sampled at 500 Hz was filtered online using 50 Hz notch filter and 2nd order Chebyshev type 2, 12 Hz lowpass filter. Epochs from -200 to 900 ms aligned to the stimuli were linearly detrended, baseline-corrected to the interval containing 200ms before the stimuli and decimated by a factor of 21, resulting in a new sampling rate 23.809 Hz. Channels of the decimated signal were then concatenated into one flat feature vector of length 92 (23 features per channel).

An LDA classifier (sklearn python library (Pedregosa et al., 2011), sklearn.discriminant analysis. Linear Discriminant Analysis) with automatic shrinkage parameter (adjusted as proposed by Ledoit and Wolf, 2003) was fitted to responses to 40 targets and 40 nontargets. Optimal decision thresholds for the classifier were calculated for each participant, based upon leave-one-out calibration and False Positive Rates.

The classifier issues a final decision based upon two conditions: either each of the stimuli was presented at least 10 times, or there were 3 consecutive decisions above the threshold for one choice and below threshold results for the other choice.

SSVEP

The SSVEP analysis is loosely based on Ajami et al. (2018). In the first step, the EEG signal was online filtered by cascade of IIR filters: 50, 100, 150, 200 Hz notch filters and Butterworth second order bandpass filter with edges 20 and 60 Hz. Filtered signal is stored in a circular buffer which returns epochs 1 second long – for classifier training each second and for feedback/communication each 0.5 seconds.

For each of the frequencies used in stimulation, a design matrix was created from 1 second long sines and cosines with the stimulation frequency fS, its first harmonic 2fS and subharmonic fS/2. Using this matrix and raw EEG signal, the LASSO model (implementation in sklearn library sklearn.linear model.coordinate descent.Lasso (Pedregosa et al., 2011), with parameters differing from default: alpha=1, max iter=1000, warm start=True, selection=cyclic, fit intercept=False) was trained for every returned buffer.

A set of candidate frequencies is chosen from the interval 15–45 Hz with step 1–2 Hz, chosen to exclude harmonics. Additionally, to each integer frequency a random number from [-0.1, 1 Hz] was added. Calibration for 2-field BCI finds the two highest frequencies, giving the largest contrast in SSVEP responses. The contrast was measured in terms of the LASSO weights. Additional threshold for these weights was set for detection of non-control state, using epochs recorded when the participant was asked to concentrate on non-blinking fields.

Every 0.5 second a 1-second signal buffer was used to compute LASSO weights. Frequency with the highest LASSO weight was selected as the action chosen in this buffer. When two consecutive buffers returned the same selected frequencies, and LASSO weights for both buffers were higher than the threshold, the choice was accepted, stimulation was stopped and the feedback for the chosen action was presented. Otherwise the system was in non-control state and stimulation was continued.

Communication efficiency measures

To assess the results of BCI communication, for each participant and paradigm we computed:

Mean answering time

where Ti – time for answering each question, counted from the start of the stimulation (after end of question) until the presentation of feedback, N – number of questions (in this study). Presented in seconds (s).

Information transfer rate

where P – probability of the correct answer, N – number of BCI classes (in this study), – mean answering time from Equation (1) in minutes. Numerator derived from work by Shannon and Elwood (1948), denominator in minutes, hence the units of reported ITR are bits/min.

Communication accuracy

Assuming everyone knew the answer to trivial questions, the units of reported ACC are %

where Ncorr – number of correctly answered questions, N – total number of questions (in this study N=20), P-value of one-sided binomial test on the amount of correct yes/no answers, assuming zero hypothesis of answers being random as a coin toss (implementation from the SciPy Python library (Jones et al., 2001), which was used to define users that were able to communicate (P<0.05) and users who were not able (Müller-Putz et al., 2008).


5. Conclusion

This paper presented new flash pattern approaches that can lead to better classification accuracies and bit rates than the conventional row/column approach. Furthermore, an adaptive classifier performed better than the non-adaptive approach. Future work could address many topics, including SAINSTs with different displays, different TTIs, effects of training, differences across different populations, peripheral stimuli, and the underlying reasons why SAINSTs impair performance. Future work should also extend this speller to more real world tasks such as controlling a laptop, writing a letter, or chatting online. This approach should also be validated with persons with special needs, such as persons with severe motor disabilities or language disorders such as dyslexia and aphasia.


Simultaneous detection of P300 and steady-state visually evoked potentials for hybrid brain-computer interface

Objective: We study the feasibility of a hybrid Brain-Computer Interface (BCI) combining simultaneous visual oddball and Steady-State Visually Evoked Potential (SSVEP) paradigms, where both types of stimuli are superimposed on a computer screen. Potentially, such a combination could result in a system being able to operate faster than a purely P300-based BCI and encode more targets than a purely SSVEP-based BCI.

Approach: We analyse the interactions between the brain responses of the two paradigms, and assess the possibility to detect simultaneously the brain activity evoked by both paradigms, in a series of 3 experiments where EEG data are analysed offline.

Main results: Despite differences in the shape of the P300 response between pure oddball and hybrid condition, we observe that the classification accuracy of this P300 response is not affected by the SSVEP stimulation. We do not observe either any effect of the oddball stimulation on the power of the SSVEP response in the frequency of stimulation. Finally results from the last experiment show the possibility of detecting both types of brain responses simultaneously and suggest not only the feasibility of such hybrid BCI but also a gain over pure oddball- and pure SSVEP-based BCIs in terms of communication rate.

Conflict of interest statement

Competing Interests: The authors have declared that no competing interests exist.


Progress in BCI in SSVEP-based Spellers

In the previous post we discussed the fundamentals of SSVEP-based BCI and the parameters that can affect their performance. In this post we will look at the design of SSVEP-based spellers, as they are one of the most common applications of BCI, considering that they are extremely useful for patients with severe motor disabilities, enabling them to communicate with others.

The performance of BCI is typically measured in terms of information transfer rate (ITR). ITR, measured as bits per trial, is computed by taking into account the accuracy of the BCI (correct command / total number of commands) , the number of possible selections (i.e. targets) and the average time for a selection. So the higher the ITR, the better. This website provides a nice overview on how to calculate ITR (https://bci-lab.hochschule-rhein-waal.de/en/itr.html). Basically it uses concepts from information theory to estimate the amount of information that is conveyed by the system.

When it comes to BCI spellers that are commonly implemented with EEG, the low SNR of EEG has limited the ITR to approximately 1 bits/second [1]. Especially the P300-based speller has very low ITR of around 0.5 bits/second [2]. Spellers based on SSVEP have shown to achieve relatively higher ITR of around 1.7 bits/second [3].

Figure 1 : ITR of the current BCI spellers. Hybrid means spellers implemented with multiple evoked potentials (example SSVEP and P300). The graph from [1] shows that the ITRs have been increasing over the years and the maximal ITR achieved is shown as Present in the graph.

The Bremen speller is one of the earliest high-speed spellers based on SSVEP. In this design, a 32-character keyboard is used along with five boxes (four arrows and a select button) as shown in the figure below.

Figure 2 : The GUI layout of Bremen speller. The letters in the keyboard are arranged according to their probability of occurrence in the English language [4].

The five boxes surrounding the keyboard, flicker at different frequencies corresponding to the command – left, right, up, down and select. By focusing on one of five flickering boxes, the subject can move the cursor to the desired letter. At the beginning of each trial, the location of the cursor is reset over the character E. Also, the characters in the keyboard are arranged according to the frequency of their usage in the English language. Imagine that the subject has to spell the word EAT. This would require the following commands. (Note that after selection of each character, the location of the cursor is reset over the character E).
• select – since the cursor is already set over the character E at the beginning, this will select the character E.
• left followed by select – This will enable the used to choose the character A.
• up followed by select – This will enable the used to choose the character T.

Thus, to spell the word EAT a minimum of 5 commands are needed (select, left, select, up, select). In the Bremen BCI speller, an audio feedback is given after every recognized command. The bottom of the screen displays the current status of the spelling. For example, in the figure above the text – PACK MY BOX WITH FIVE DOZEN LIQUOR is shown. Also, it has to be noted that the navigation cannot move over the layout boundaries, i.e., you cannot move from L to H by choosing the command up. Improvements to the Bremen speller include addition of a built-in dictionary to predict the words as well as giving feedback to the user by increasing the size of the box, when the SSVEP signal increased, notifying the user that the selection was about to be made [5]. Studies have reported an average ITR of 0.43 bits/second, with an accuracy of 93.27%, which indicates a competitive performance, considering that patients with neural malfunctions participated in the study [5]. Several other high-speed spellers have been also been proposed (see [5] for detailed review) that include the row column design similar to the P300 speller as well as stimulation matrix resembling an alphanumerical keyboard (see Figure 2).

By using advanced signal processing methods, considerable improvement in the ITR can be achieved. For example in [1], it was shown that the SSVEP-based speller can achieve an ITR of 4.45 bits/second in online spelling tasks with healthy participants, using methods to precisely encode the frequency and phase of the stimulation signals in single-trial SSVEPs, with short stimulation duration. However, their method assumes visual latency across trials to be stable, in order to jointly code for frequency and phase of the stimulus. This assumption could be challenged, as shown in [6] that trial-to-trial variability of both amplitude and latency cannot be ignored.

Figure 3 : High speed-BCI speller proposed in [1] that utilizes techniques to precisely encode the frequency and phase of the stimulation signals to achieve high ITR with single-trial SSVEPs.

References

[1] Chen, Xiaogang, et al. “High-speed spelling with a noninvasive brain–computer interface.” Proceedings of the national academy of sciences 112.44 (2015): E6058-E6067.
[2] Farwell LA, Donchin E (1988) Talking off the top of your head: Toward a mental prosthesis utilizing event-related brain potentials. Electroencephalogr Clin Neurophysiol 70(6):510–523.
[3] Chen X, Chen Z, Gao S, Gao X (2014) A high-ITR SSVEP-based BCI speller. Brain-Comp Interfaces 1(3–4):181–191.
[4] Rezeika, Aya, et al. “Brain–computer interface spellers: A review.” Brain sciences 8.4 (2018): 57.
[5] Volosyak I., Cecotti H., Valbuena D., Graser A. Evaluation of the Bremen SSVEP based BCI in real world conditions Proceedings of the 2009 IEEE International Conference on Rehabilitation Robotics Kyoto, Japan. 23–26 June 2009 pp. 322–331.
[6] Truccolo, Wilson A., et al. “Trial-to-trial variability of cortical evoked responses: implications for the analysis of functional connectivity.” Clinical neurophysiology 113.2 (2002): 206-226.

3 thoughts on &ldquo Progress in BCI in SSVEP-based Spellers &rdquo

Narayan, thanks for your article.

I do believe that cVEP (code based VEP) spellers have decidedly taken the lead in ITR in recent years. Below is a quite remarkable open source cVEP speller,

https://iopscience.iop.org/article/10.1088/1741-2552/abecef/meta
“From full calibration to zero training for a code-modulated visual evoked potentials brain computer interface”

Well, your comment system appears to LOSE all the paragraph formatting I placed in the above comment. A shame, since it is now 10 times harder to read the links.


Current Status of P300 BCIs

Although the P300 BCI was introduced in 1988 (Farwell and Donchin, 1988), it initially received very little attention. From 1988 to 2000, there were no P300 BCI peer-reviewed papers (Donchin et al., 2000). The next five years saw only a modest increase in P300 BCI articles, which often relied on offline analyses, such as analyzing other groups' data from the 2003 BCI Data Analysis Competition (Allison and Pineda, 2003 Xu et al., 2004).

However, in the past few years, P300 BCIs have clearly emerged as one of the main BCI categories. P300 BCIs have consistently exhibited several appealing features—they are relatively fast, effective for most users, straightforward, and require practically no training. Recent work has shown that P300 BCIs can be used for a wide range of different functions and can work with disabled users in home settings (K࿋ler et al., 2009 Sellers et al., 2010 Kleih et al., 2012), although some concerns about gaze shifting have emerged (Allison et al., 2008 Brunner et al., 2010 Treder and Blankertz, 2010). In addition, new paradigms for eliciting the P300 have been introduced (Fazel-Rezai and Abhari, 2009 Townsend et al., 2010), and new ways to flash or otherwise change stimuli could enhance ERPs and improve classification (Kaufmann et al., 2011 Jin et al., 2012). Overall, it is likely that P300 BCIs will remain prominent in the foreseeable future, but will probably grow further and further away from the canonical 6 × 6 matrix with single row and column flashes described in the first two P300 BCI articles.

P300 Papers Submitted in Journals Since 2000

The past few years have seen a strong increase in P300 BCI research. Figure 1 shows the number of peer-reviewed journal publications that were identified via PubMed and Scopus search engines from 2000 to 2010 with the phrase “[(P300 OR P3) AND (BCI OR Brain Computer Interface)].” Conference proceedings were removed from the search. These articles reflect numerous novel directions. In addition to improving information transfer rate, there has been considerable success extending P300 BCIs to new tasks, paradigms, and applications.

Figure 1. Number of published journal papers in PubMed and Scopus from 2000 to 2010 when “[(P300 OR P3) AND (BCI OR Brain Computer Interface)]” keyword was used.

2010 BCI Award Submission Statistics

To highlight trends and developments of BCI technology, g.tec (Medical Engineering GMBH, Austria) began to sponsor the Annual BCI Award in 2010. Fifty-seven projects were submitted to the BCI Award 2010 and an international committee nominated the 10 top-ranked candidates (Guger, 2011).

Table 1 categorizes the BCI Award 2010 nominees into utilized control signals and application areas. Of the eight nominated projects that used EEG as input signal, six utilized the N200/P300 response. N200 is a negative peak in the ERP that appears about 200 ms after a stimulus onset (Hong et al., 2009). The committees decision to select a number of P300 BCI paradigms is reflected by several reasons: (1) the P300 response is easy to measure and non-invasive, (2) it requires less than 10 min of training, (3) it works with the majority of subjects including those with the severe neurological disease, and (4) gives a goal-oriented control signal that is especially suited for spelling and control application where no continuous control signal is needed (e.g., Internet surfing, painting). All the spelling/Internet/art applications selected by the BCI Award committee were controlled with the P3 N200/P300 strategy. The two other projects used motor imagery (MI) to generate a continuous control signal. Both MI projects used the BCI system for the activation of the sensorimotor cortex for stroke rehabilitation, which is not possible with N200/P300- or SSVEP-based BCI systems. In the 2010 competition, 40.4% of 57 submissions used MI, 29.8% used P300 and 8.9% used the SSVEP principles, and the rest used other modalities.

Table 1. Categorization of the 10 BCI Award nominees.


Mental fatigue level detection based on event related and visual evoked potentials features fusion in virtual indoor environment

The purpose of this work is to set up a model that can estimate the mental fatigue of users based on the fusion of relevant features extracted from Positive 300 (P300) and steady state visual evoked potentials (SSVEP) measured by electroencephalogram. To this end, an experimental protocol describes the induction of P300, SSVEP and mental workload (which leads to mental fatigue by varying time-on-task) in different scenarios where environmental artifacts are controlled (obstacles number, obstacles velocities, ambient luminosity). Ten subjects took part in the experiment (with two suffering from cerebral palsy). Their mission is to navigate along a corridor from a starting point A to a goal point B where specific flickering stimuli are introduced to perform the P300 task. On the other hand, SSVEP task is elicited thanks to 10 Hz flickering lights. Correlated features are considered as inputs to fusion block which estimates mental workload. In order to deal with uncertainties and heterogeneity of P300 and SSVEP features, Dempster–Shafer (D–S) evidential reasoning is introduced. As the goal is to assess the reliability for the estimation of mental fatigue levels, D–S is compared to multi layer perception and linear discriminant analysis. The results show that D–S globally outperforms the other classifiers (although its performance significantly decreases between healthy and palsied groups). Finally we discuss the feasibility of such a fusion proposal in real life situation.

This is a preview of subscription content, access via your institution.


Watch the video: Visual Evoked Potentials 101 (August 2022).