1. Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China 2. Beijing Institute of Machinery and Equipment, Beijing 100039, China
A novel EEGNet variation based on the fusion of the Inception and attention mechanism modules was proposed, called IncepA-EEGNet, in order to achieve more efficient P300 signal feature extraction. Convolutional layers with different receptive fields were connected in parallel. The network’s ability to extract and express EEG signals were enhanced. Then the attention mechanism was introduced to assign weights to the features of different filters, and important information was extracted from the P300 signal. The validation experiment was conducted on two subjects of BCI Competition III dataset II. Results showed that the IncepA-EEGNet recognition accuracy reached 75.5% after just 5 epochs compared with other deep learning models. The information transmission rate was up to 33.44 bits/min on subject B after 3 epochs. These experimental results demonstrate that the IncepA-EEGNet effectively improves the recognition accuracy of the P300 signal, reduces the time of repeated trials, and enhances the applicability of the P300 speller.
Meng XU,Dan WANG,Zhi-yuan LI,Yuan-fang CHEN. IncepA-EEGNet: P300 signal detection method based on fusion of Inception network and attention mechanism. Journal of ZheJiang University (Engineering Science), 2022, 56(4): 745-753, 782.
Fig.5SE attention mechanism combined with Inception-v1 module
受试者
训练集样本数
测试集样本数
目标
非目标
目标
非目标
受试者A
2550
12750
3000
15000
受试者B
2550
12750
3000
15000
Tab.1Datasets’s information of P300 speller
Fig.6Overall framework for P300 speller character recognition
受试者
K
卷积核大小
Acc
R
P
F1
A
8
(8/4/2,1)
0.7323
0.6630
0.3432
0.4523
A
16
(16/8/4,1)
0.7384
0.6547
0.3485
0.4548
A
32
(32/16/8,1)
0.7425
0.6486
0.3520
0.4564
A
64
(64/32/16,1)
0.7314
0.6677
0.3430
0.4532
A
128
(128/64/32,1)
0.7553
0.6457
0.3670
0.4679
A
160
(160/80/40,1)
0.7514
0.6210
0.3582
0.4543
B
8
(8/4/2,1)
0.7817
0.7034
0.4122
0.5149
B
16
(16/8/4,1)
0.7855
0.7186
0.4168
0.5276
B
32
(32/16/8,1)
0.7848
0.7143
0.4154
0.5253
B
64
(64/32/16,1)
0.7899
0.6993
0.4215
0.5260
B
128
(128/64/32,1)
0.7914
0.7250
0.4261
0.5367
B
160
(160/80/40,1)
0.7889
0.6987
0.4199
0.5245
Tab.2Classification results using different convolution kernel parameters K on IncepA-EEGNet
受试者
r
Acc
R
P
F1
A
1
0.7458
0.6473
0.3557
0.4592
A
3
0.7553
0.6457
0.3670
0.4679
A
9
0.7425
0.6486
0.3520
0.4610
A
12
0.7314
0.6683
0.3430
0.4533
B
1
0.7869
0.7227
0.4193
0.5307
B
3
0.7914
0.7250
0.4261
0.5367
B
9
0.7954
0.6880
0.4291
0.5285
B
12
0.7829
0.7133
0.4125
0.5227
Tab.3Classification results of different descending coefficients used by attention mechanism
Fig.7Training loss and test accuracy on IncepA-EEGNet
添加模块
Acc
受试者
CNN-1
MCNN-1
MCNN-3
EEGNet
基础网络(Net)
A
0.7037
0.6899
0.7038
0.7065
基础网络(Net)
B
0.7065
0.6912
0.7037
0.7266
Net+Attention
A
0.7092
0.6906
0.7091
0.7141
Net+Attention
B
0.7185
0.7154
0.7192
0.7399
Net+Inception-v1
A
0.7100
0.6965
0.7103
0.7174
Net+Inception-v1
B
0.7222
0.7276
0.7203
0.7476
Net+Attention +Inception-v1
A
0.7186
0.7084
0.7258
0.7553
Net+Attention +Inception-v1
B
0.7454
0.7384
0.7478
0.7914
Tab.4Impact of adding sub-modules to different CNN networks on classification accuracy
方法
受试者
Acc
R
P
F1
CNN-1[10]
A
0.7037
0.6737
0.3170
0.4311
CNN-1[10]
B
0.7065
0.6783
0.4073
0.5090
MCNN-1[10]
A
0.6899
0.6903
0.3085
0.4260
MCNN-1[10]
B
0.6912
0.7340
0.3833
0.5034
MCNN-3[10]
A
0.7038
0.6743
0.3172
0.4314
MCNN-3[10]
B
0.7037
0.6923
0.4089
0.5141
EEGNet[20]
A
0.7065
0.6460
0.3147
0.4232
EEGNet[20]
B
0.7266
0.6950
0.4214
0.4587
BN3[17]
A
0.7513
0.6133
0.3607
0.4605
BN3[17]
B
0.7902
0.6947
0.4214
0.5246
IncepA-EEGNet
A
0.7553
0.6456
0.3676
0.4679
IncepA-EEGNet
B
0.7914
0.7250
0.4261
0.5367
Tab.5Comparison of IncepA-EEGNet’s performance with other deep learning methods on P300 signal classification
Fig.8Comparison of information transfer rate of IncepA-EEGNet model with other methods on subject A and subject B
方法
受试者
Pc/%
n = 1
n = 2
n = 3
n = 4
n = 5
n = 6
n = 7
n = 8
n = 9
n = 10
n = 11
n = 12
n = 13
n = 14
n = 15
CNN-1[10]
A
16
33
47
52
61
65
77
78
85
86
90
91
91
93
97
CNN-1[10]
B
35
52
59
68
79
81
82
89
92
91
91
90
91
92
92
MCNN-1[10]
A
18
31
50
54
61
68
76
76
79
82
89
92
91
93
97
MCNN-1[10]
B
39
55
62
64
77
79
86
92
91
92
95
95
95
94
94
MCNN-3[10]
A
17
35
50
55
63
67
78
79
84
85
91
90
92
94
97
MCNN-3[10]
B
34
56
60
68
74
80
82
89
90
90
91
88
90
91
92
BN3[17]
A
22
39
58
67
73
75
79
81
82
86
89
92
94
96
98
BN3[17]
B
47
59
70
73
76
82
84
91
94
95
95
95
94
94
95
EEGNet[20]
A
18
33
46
60
68
70
82
82
83
85
88
90
91
96
99
EEGNet[20]
B
39
49
56
65
76
80
85
87
89
89
90
90
90
92
93
1D-CapsNet-64[18]
A
21
32
45
53
60
68
76
83
85
84
82
88
94
96
98
1D-CapsNet-64[18]
B
48
54
60
66
75
81
81
86
87
93
93
93
92
93
94
CM-CW-CNN-ESVM[19]
A
22
32
55
59
64
70
74
78
81
86
86
90
91
94
99
CM-CW-CNN-ESVM[19]
B
37
58
70
72
80
86
86
89
93
95
95
97
97
98
99
IncepA-EEGNet
A
19
34
47
62
70
71
84
83
85
89
92
93
94
96
100
IncepA-EEGNet
B
41
59
73
77
81
85
88
90
92
95
95
95
95
95
95
Tab.6Character recognition rate of IncepA-EEGNet model and other methods
受试者
字符编号
期望字符
输出字符
A
16
P
Q
B
24
Q
P
B
39
V
W
B
10
Z
H
Tab.7Confusion of character recognition on P300 speller
方法
参数量
EEGNet
5428
EEGNet+Attention
8969
EEGNet+Inception-v1
12742
IncepA-EEGNet
22970
Tab.8Number of trainable parameters of EEGNet after adding different sub-modules
[1]
IKEGAMI S, TAKANO K, SAEKI N, et al Operation of a P300-based brain–computer interface by individuals with cervical spinal cord injury[J]. Clinical Neurophysiology, 2011, 122 (5): 991- 996
doi: 10.1016/j.clinph.2010.08.021
[2]
RICCIO A, SCHETTINI F, SIMIONE L, et al On the relationship between attention processing and P300-based brain computer interface control in amyotrophic lateral sclerosis[J]. Frontiers in Human Neuroscience, 2018, 12 (165): 1- 10
FARWELL L A, DONCHIN E Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials[J]. Electroencephalography and Clinical Neurophysiology, 1988, 70 (6): 510- 523
doi: 10.1016/0013-4694(88)90149-6
张楠楠. 脑机接口中的视觉刺激设计与优化方法研究[D]. 长沙: 国防科技大学, 2019: 84. ZHANG Nan-nan. Research on the design and optimization method of visual stimuli in brain-computer interface [D]. Changsha: National University of Defense Technology, 2019: 84.
[7]
XU M, CHEN L, ZHANG L, et al A visual parallel-BCI speller based on the time–frequency coding strategy[J]. Journal of Neural Engineering, 2014, 11 (2): 026014.1- 026014.11
[8]
RAKOTOMAMONJY A, GUIGUE V BCI competition III: dataset II-ensemble of SVMs for BCI P300 speller[J]. IEEE Transactions on Biomedical Engineering, 2008, 55 (3): 1147- 1154
doi: 10.1109/TBME.2008.915728
[9]
KRUSIENSKI D J, SELLERS E W, CABESTAING F, et al A comparison of classification techniques for the P300 speller[J]. Journal of Neural Engineering, 2006, 3 (4): 299.1- 299.13
[10]
CECOTTI H, GRSER A Convolutional neural networks for p300 detection with application to brain computer interfaces[J]. IEEE Transactions on Software Engineering, 2011, 33 (3): 433- 445
[11]
ZHANG C, KIM Y K, ESKANDARIAN A EEG-inception: an accurate and robust end-to-end neural network for EEG-based motor imagery classification[J]. Journal of Neural Engineering, 2021, 18 (4): 046014.1- 046014.16
[12]
AMIN S U, ALTAHERI H, MUHAMMAD G, et al. Attention based Inception model for robust EEG motor imagery classification [C]// 2021 IEEE International Instrumentation and Measurement Technology Conference. Glasgow: IEEE, 2021: 1-6.
[13]
LI Y, LIU Y, CUI W G, et al Epileptic seizure detection in EEG signals using a unified temporal-spectral squeeze-and-excitation network[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2020, 28 (4): 782- 794
doi: 10.1109/TNSRE.2020.2973434
[14]
贾子钰, 林友芳, 刘天航, 等 基于多尺度特征提取与挤压激励模型的运动想象分类方法[J]. 计算机研究与发展, 2020, 57 (12): 2481- 2489 JIA Zi-yu, LIN You-fang, LIU Tian-hang, et al Motor imagery classification based on multiscale feature extraction and squeeze-excitation model[J]. Journal of Computer Research and Development, 2020, 57 (12): 2481- 2489
doi: 10.7544/issn1000-1239.2020.20200723
[15]
RIVET B, SOULOUMIAC A, ATTINA V, et al xDAWN algorithm to enhance evoked potentials: application to brain–computer interface[J]. IEEE Transactions on Biomedical Engineering, 2009, 56 (8): 2035- 2043
doi: 10.1109/TBME.2009.2012869
[16]
XIAO X, XU M, JIN J, et al Discriminative canonical pattern matching for single-trial classification of ERP components[J]. IEEE Transactions on Biomedical Engineering, 2019, 67 (8): 2266- 2275
[17]
LIU M, WU W, GU Z, et al Deep learning based on batch normalization for P300 signal detection[J]. Neurocomputing, 2018, 275: 288- 297
doi: 10.1016/j.neucom.2017.08.039
[18]
LIU X, XIE Q, LV J, et al P300 event-related potential detection using one-dimensional convolutional capsule networks[J]. Expert Systems with Applications, 2021, 174 (15): 114701.1- 114701.12
[19]
KUNDU S, ARI S P300 based character recognition using convolutional neural network and support vector machine[J]. Biomedical Signal Processing and Control, 2020, 55: 101645.1- 101645.7
[20]
LAWHERN V J, SOLON A J, WAYTOWICH N R, et al EEGNet: a compact convolutional network for EEG-based brain-computer interfaces[J]. Journal of Neural Engineering, 2016, 15 (5): 056013.1- 056013.17
[21]
SZEGEDY C, LIU W, JIA Y, et al. Going deeper with convolutions [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 1-9.
[22]
HU J, SHEN L, SUN G. Squeeze-and-excitation networks [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 7132-7141.
[23]
WANG Q, WU B, ZHU P, et al. ECA-Net: efficient channel attention for deep convolutional neural networks [C]// CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 11531-11539.
[24]
XIAO J, LIN Q, YU T, et al. A BCI system for assisting visual fixation assessment in behavioral evaluation of patients with disorders of consciousness [C]// 2017 8th International IEEE/EMBS Conference on Neural Engineering. Shanghai: IEEE, 2017: 399-402.
[25]
LIU L, WU F X, WANG Y P, et al Multi-receptive-field CNN for semantic segmentation of medical images[J]. IEEE Journal of Biomedical and Health Informatics, 2020, 24 (11): 3215- 3225
doi: 10.1109/JBHI.2020.3016306
[26]
WANG H, XU J, YAN R, et al A new intelligent bearing fault diagnosis method using SDP representation and SE-CNN[J]. IEEE Transactions on Instrumentation and Measurement, 2019, 69 (5): 2377- 2389
[27]
LUO Y, LU B L. EEG data augmentation for emotion recognition using a conditional Wasserstein GAN [C]// 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Honolulu: IEEE, 2018: 2535-2538.
[28]
LEE T, KIM M, KIM S P. Data augmentation effects using borderline-SMOTE on classification of a P300-based BCI [C]// 2020 8th International Winter Conference on Brain-Computer Interface. Gangwon: IEEE, 2020
