|
|
|
| Classification of group speech imagined EEG signals based on attention mechanism and deep learning |
Yifan ZHOU1( ),Lingwei ZHANG1,2,Zhengdong ZHOU1,*(),Zhi CAI1,Mengyao YUAN1,Xiaoxi YUAN1,Zeyi YANG1 |
1. State Key Laboratory of Mechanics and Control for Aerospace Structures, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, China
2. VeriSilicon Holdings (Nanjing) Co. Ltd, Nanjing 210000, China |
|
|
|
Abstract A classification method based on convolutional block attention module (CBAM) and Inception-V4 convolutional neural network was proposed to improve the classification accuracy of group EEG signals of imagined speech. CBAM was used to emphasize significant localized areas and extract distinctive features from the output feature map of convolutional neural network (CNN), so as to improve the classification performance of group EEG signals of imagined speech. The group EEG signals of imagined speech were converted into time-frequency images by short-time Fourier transform, then the images were used to train the Inception-V4 network incorporating with CBAM. Experiments on an open-accessed dataset showed that the proposed method achieved an accuracy of 52.2% in classifying six types of short words, which was 4.1 percentage points higher than that with Inception-V4 and was 5.9 percentage points higher than that with VGG-16. Furthermore, the training time can be reduced greatly with transfer learning.
|
|
Received: 17 October 2023
Published: 25 November 2024
|
|
|
| Fund: 中国航空研究院首批揭榜挂帅项目（F2021109）；上海航天科技创新基金资助项目 (SAST2019-121）；南京航空航天大学科研与实践创新计划资助项目（xcxjh20210104）；江苏高校优势学科建设工程资助项目（PAPD）. |
|
Corresponding Authors:
Zhengdong ZHOU
E-mail: sz2201046@nuaa.edu.cn;zzd_msc@nuaa.edu.cn
|
基于注意力机制和深度学习的群体语言想象脑电信号分类
为了提高群体语言想象脑电信号的分类准确率，提出基于卷积块注意力模块（CBAM）和Inception-V4卷积神经网络的分类方法，其中CBAM被用于关注重要的局部区域，从卷积神经网络（CNN）输出的特征图中提取更加独特的特征，从而提升群体语言想象脑电信号的分类性能. 该方法首先利用短时傅里叶变换将群体语言想象脑电信号转换为时频图，然后使用这些图片对融合了CBAM机制的Inception-V4网络进行训练. 开源数据集上的实验结果表明，所提出的方法使得6类短词的分类准确率达到了52.2%，与基于Inception-V4的分类方法相比，分类准确率提高了4.1个百分点，与基于VGG-16的分类方法相比，分类准确率提高了5.9个百分点. 使用迁移学习也能够大幅缩短训练所需的时间.
关键词：
脑-机接口,
脑电图,
语言想象,
深度学习,
注意力机制
|
|
| [18] |
WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module [C]// Proceedings of the European Conference on Computer Vision . Munich: Springer Cham, 2018: 3−19.
|
|
|
| [19] |
JADERBERG M, SIMONYAN K, ZISSERMAN A, et al. Spatial transformer networks [C]// Proceedings of the 28th International Conference on Neural Information Processing Systems . Montreal: MIT Press, 2015: 2017−2025.
|
|
|
| [20] |
WANG Q, WU B, ZHU P, et al. ECA-Net: efficient channel attention for deep convolutional neural networks [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle: IEEE, 2020: 11534−11542.
|
|
|
| [21] |
SELVARAJU R R, COGSWELL M, DAS A, et al. Grad-cam: visual explanations from deep networks via gradient-based localization [C]// Proceedings of the IEEE International Conference on Computer Vision . Venice: IEEE, 2017: 618−626.
|
|
|
| [22] |
BISWAS S, SINHA R. Lateralization of brain during EEG based covert speech classification [C]// 2018 15th IEEE India Council International Conference . Coimbatore: IEEE, 2018: 1−5.
|
|
|
| [23] |
LEE D Y, LEE M, LEE S W. Classification of imagined speech using Siamese neural network [C]// 2020 IEEE International Conference on Systems, Man, and Cybernetics . Toronto: IEEE, 2020: 2979−2984.
|
|
|
| [24] |
COONEY C, KORIK A, FOLLI R, et al Evaluation of hyperparameter optimization in machine and deep learning methods for decoding imagined speech EEG[J]. Sensors, 2020, 20 (16): 4629
doi: 10.3390/s20164629
|
|
|
| [25] |
DAGDEVIR E, TOKMAKCI M Determination of effective signal processing stages for brain computer interface on BCI competition IV data set 2b: a review study[J]. IETE Journal of Research, 2023, 69 (6): 3144- 3155
doi: 10.1080/03772063.2021.1914204
|
|
|
| [1] |
NGUYEN C H, KARAVAS G K, ARTEMIADIS P Inferring imagined speech using EEG signals: a new approach using Riemannian manifold features[J]. Journal of Neural Engineering, 2017, 15 (1): 016002
|
|
|
| [2] |
ROYER A S, DOUD A J, ROSE M L EEG control of a virtual helicopter in 3-dimensional space using intelligent control strategies[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2010, 18 (6): 581- 589
doi: 10.1109/TNSRE.2010.2077654
|
|
|
| [3] |
BALAM V P, CHINARA S Statistical channel selection method for detecting drowsiness through single-channel EEG-based BCI system[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70 (7): 1- 9
|
|
|
| [4] |
KHARE S K, BAJAJ V, SINHA G R Adaptive tunable Q wavelet transform-based emotion identification[J]. IEEE Transactions on Instrumentation and Measurement, 2020, 69 (12): 9609- 9617
doi: 10.1109/TIM.2020.3006611
|
|
|
| [5] |
KAMBLE K S, SENGUPTA J Ensemble machine learning-based affective computing for emotion recognition using dual-decomposed EEG signals[J]. IEEE Sensors Journal, 2021, 22 (3): 2496- 2507
|
|
|
| [6] |
WIERZGAŁA P, ZAPAŁA D, WOJCIK G M Most popular signal processing methods in motor-imagery BCI: a review and meta-analysis[J]. Frontiers in Neuroinformatics, 2018, 12 (11): 78
|
|
|
| [7] |
LEE S H, LEE M, LEE S W. EEG representations of spatial and temporal features in imagined speech and overt speech [C]// Pattern Recognition: 5th Asian Conference . Auckland: Springer International Publishing, 2020: 387−400.
|
|
|
| [8] |
刘艳鹏, 龚安民, 丁鹏, 等 基于言语想象的脑机交互关键技术[J]. 生物医学工程学杂志, 2022, 39 (3): 596- 611
LIU Yanpeng, GONG Anmin, DING Peng, et al Key technology of brain-computer interaction based on speech imagery[J]. Journal of Biomedical Engineering, 2022, 39 (3): 596- 611
doi: 10.7507/1001-5515.202107018
|
|
|
| [9] |
VAN DEN BERG B, VAN DONKELAAR S, ALIMARDANI M. Inner speech classification using EEG signals: a deep learning approach [C]// 2021 IEEE 2nd International Conference on Human-Machine Systems . Magdeburg: IEEE, 2021: 1−4.
|
|
|
| [10] |
RUSNAC A L, GRIGORE O CNN architectures and feature extraction methods for EEG imaginary speech recognition[J]. Sensors, 2022, 22 (13): 4679- 4698
doi: 10.3390/s22134679
|
|
|
| [11] |
LEE D Y, LEE M, LEE S W Decoding imagined speech based on deep metric learning for intuitive BCI communication[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2021, 29 (7): 1363- 1374
|
|
|
| [12] |
CORETTO G A P, GAREIS I E, RUFINER H L. Open access database of EEG signals recorded during imagined speech [C]// 12th International Symposium on Medical Information Processing and Analysis . Munich: SPIE, 2017: 1016002.
|
|
|
| [13] |
CAO Y, OOSTENVELD R, ALDAY P M, et al Are alpha and beta oscillations spatially dissociated over the cortex in context-driven spoken-word production?[J]. Psychophysiology, 2022, 59 (6): e13999
doi: 10.1111/psyp.13999
|
|
|
| [14] |
WEI X, SUN T, ZHENG L, et al. Diagnosis of loose core fault in saturable reactor of thyristor valve based on vibration signal time-frequency analysis and CNN [C]// 5th International Conference on Energy Systems and Electrical Power . Changsha: Journal of Physics: Conference Series, 2023, 2584(1): 012079.
|
|
|
| [15] |
PODDER P, KHAN T Z, KHAN M H, et al Comparative performance analysis of hamming, hanning and blackman window[J]. International Journal of Computer Applications, 2014, 96 (18): 1- 7
doi: 10.5120/16891-6927
|
|
|
| [16] |
SZEGEDY C, IOFFE S, VANHOUCKE V, et al. Inception-V4, inception-resnet and the impact of residual connections on learning [C]// Proceedings of the AAAI Conference on Artificial Intelligence . San Francisco: AAAI, 2017, 31(1): 4278−4284.
|
|
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
| |
Shared |
|
|
|
|
| |
Discussed |
|
|
|
|
