Please wait a minute...
浙江大学学报(工学版)
浙江大学学报(工学版)  2023, Vol. 57 Issue (4): 726-734    DOI: 10.3785/j.issn.1008-973X.2023.04.010
自动化技术、计算机技术     
基于特征融合的语言想象脑电信号分类
张灵维(),周正东*(),许云飞,王嘉文,吉文韬,宋泽峰
南京航空航天大学 航空学院，江苏 南京 210016
Classification of imagined speech EEG signals based on feature fusion
Ling-wei ZHANG(),Zheng-dong ZHOU*(),Yun-fei XU,Jia-wen WANG,Wen-tao JI,Ze-feng SONG
College of Aeronautics, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, China
 全文: PDF(1356 KB)   HTML
摘要：

为了提高语言想象脑-机接口（BCI）控制任务的准确率, 提出融合离散小波变换（DWT）与经验模态分解（EMD）的语言想象脑电信号特征提取与分类方法. 该方法将原始语言想象脑电信号分别进行离散小波变换与经验模态分解，提取分解后各通道信号的特征并进行融合，运用径向基核函数支持向量机（SVM）对语言想象脑电信号进行分类. 实验结果表明，提出的方法使得语言想象脑电信号分类的平均准确率达到82.46%，与基于离散小波变换的脑电信号分类方法相比，分类准确率提高了20.77%，与基于经验模态分解的脑电信号分类方法相比，分类准确率提高了21.12%. 该方法能够有效提高语言想象脑电信号分类的准确率，对于语言想象脑-机接口的实际应用具有重要价值.
关键词： 脑-机接口（BCI）;  脑电图;  语言想象;  离散小波变换（DWT）;  经验模态分解（EMD）    
Abstract:

A feature extraction and classification method of imagined speech electroencephalogram (EEG) signals was proposed by combining discrete wavelet transform (DWT) and empirical mode decomposition (EMD) in order to improve the accuracy of imagined speech brain-computer interface (BCI) control task. DWT and EMD were applied to the original imagined speech EEG signals respectively, and the features of the signal of each channel were extracted and fused. Then the RBF support vector machine (SVM) was used to classify the imagined speech EEG signals. The experimental results show that the classification accuracy can achieve an average by 82.46% with the proposed method, which is 20.77% higher than that with the DWT method, and 21.12% higher than that with the EMD method. The proposed method can effectively improve the classification accuracy of imagined speech EEG signals, and is of great value to the practical application of imagined speech BCI.
Key words: brain-computer interface (BCI)    electroencephalogram    imagined speech    discrete wavelet transform (DWT)    empirical mode decomposition (EMD)
收稿日期: 2022-06-22 出版日期: 2023-04-21
CLC:  TP 391  
基金资助: 中国航空研究院首批揭榜挂帅项目（F2021109）；上海航天科技创新基金资助项目 (SAST2019-121）；南京航空航天大学科研与实践创新计划资助项目（xcxjh20210104）；江苏高校优势学科建设工程资助项目（PAPD）
通讯作者: 周正东     E-mail: nuaazlw@nuaa.edu.cn;zzd_msc@nuaa.edu.cn
作者简介: 张灵维（1998—），男，硕士生，从事脑-机接口的研究. orcid.org/0000-0003-4731-1570. E-mail： nuaazlw@nuaa.edu.cn
服务　　
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章　　
张灵维
周正东
许云飞
王嘉文
吉文韬
宋泽峰

引用本文:

张灵维,周正东,许云飞,王嘉文,吉文韬,宋泽峰. 基于特征融合的语言想象脑电信号分类[J]. 浙江大学学报(工学版), 2023, 57(4): 726-734.

Ling-wei ZHANG,Zheng-dong ZHOU,Yun-fei XU,Jia-wen WANG,Wen-tao JI,Ze-feng SONG. Classification of imagined speech EEG signals based on feature fusion. Journal of ZheJiang University (Engineering Science), 2023, 57(4): 726-734.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2023.04.010    或    https://www.zjujournals.com/eng/CN/Y2023/V57/I4/726

图 1  基于特征融合的语言想象脑电信号分类方法流程图
图 2  用于获取脑电信号的电极位置
图 3  离散小波变换的原理
图 4  离散小波变换分解EEG信号
图 5  经验模态分解EEG信号
图 6  EEG信号特征提取与融合的流程图
图 7  基于SVM的EEG信号分类准确率
图 8  基于KNN的EEG信号分类准确率
研究方法 方法 $\bar A $/%
文献[20]方法 DWT+随机森林 19.60
文献[31]方法 张量分解法 59.70
文献[32]方法 卷积神经网络 35.68
文献[33]方法 Siamese神经网络 31.40
本文方法1 DWT+SVM 61.69
本文方法2 EMD+SVM 61.34
本文方法3 DWT-EMD+SVM 82.46
本文方法4 DWT-EMD+KNN 64.57
表 1  语言想象EEG信号分类准确率的比较
1 VAUGHAN T, HEETDERKS W, TREJO L, et al Brain-computer interface technology: a review of the second international meeting[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2003, 11 (2): 94- 109
doi: 10.1109/TNSRE.2003.814799
2 PUTZE F, SCHULTZ T Adaptive cognitive technical systems[J]. Journal of Neuroscience Methods, 2014, (234): 108- 115
3 LEE S H, LEE M, LEE S W. EEG representations of spatial and temporal features in imagined speech and overt speech [C]// Asian Conference on Pattern Recognition. Cham: Springer, 2019: 387-400.
4 PIOTR W, DARIUSZ Z, GRZEGORZ M, et al Most popular signal processing methods in motor-imagery BCI: a review and meta-analysis[J]. Frontiers in Neuroinformatics, 2018, (12): 78
5 AMIRI S, RABBI A, AZINFAR L, et al. A review of P300, SSVEP, and hybrid P300/SSVEP brain-computer interface systems [M]// Brain-computer interface systems: recent progress and future prospects. Fargo: Intech, 2013.
6 ROSENFELD J, HU X, LABKOVSKY E, et al Review of recent studies and issues regarding the P300-based complex trial protocol for detection of concealed information[J]. International Journal of Psychophysiology, 2013, 90 (2): 118- 134
doi: 10.1016/j.ijpsycho.2013.08.012
7 于淑月, 李想, 于功敬, 等 脑机接口技术的发展与展望[J]. 计算机测量与控制, 2019, (10): 5- 12
YU Shu-yue, LI Xiang, YU Gong-jing, et al Development and prospect of brain-computer interface technology[J]. Computer Measurement and Control, 2019, (10): 5- 12
8 WESTER M. Unspoken speech-speech recognition based on electroencephalography [D]. Karlsruher: University of Karlsruhe, 2006.
9 TORRES A, REYES A, VILLASENOR L, et al Implementing a fuzzy inference system in a multi-objective EEG channel selection model for imagined speech classification[J]. Expert Systems with Applications, 2016, (59): 1- 12
10 QURESHI I, MIN B, PARK H, et al Multiclass classification of word imagination speech with hybrid connectivity features[J]. IEEE Transactions on Biomedical Engineering, 2017, 65 (10): 2168- 2177
11 HASHIM N, ALI A, MOHD N. Word-based classification of imagined speech using EEG [C]// International Conference on Computational Science and Technology. Singapore: Springer, 2017: 195-204.
12 LEE S, LEE M, JEONG H, et al. Towards an EEG-based intuitive BCI communication system using imagined speech and visual imagery [C]// 2019 IEEE International Conference on Systems, Man and Cybernetics. Bari: IEEE, 2019.
13 SRIRAAM N, RAGHU S Classification of focal and non focal epileptic seizures using multi-features and SVM classifier[J]. Journal of Medical Systems, 2017, 41 (10): 160
doi: 10.1007/s10916-017-0800-x
14 MA L, ZHANG T, DONG C A novel ECG data compression method using adaptive Fourier decomposition with security guarantee in e-health applications[J]. IEEE Journal of Biomedical and Health Informatics, 2015, 19 (3): 986- 994
doi: 10.1109/JBHI.2014.2357841
15 ZENG W, LI M, YUAN C, et al Identification of epileptic seizures in EEG signals using time-scale decomposition (ITD), discrete wavelet transform (DWT), phase space reconstruction (PSR) and neural networks[J]. Artificial Intelligence Review, 2020, 53 (4): 3059- 3088
doi: 10.1007/s10462-019-09755-y
16 HUANG E, SHEN Z, LONG R, et al The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis[J]. Proceedings Mathematical Physical and Engineering Sciences, 1998, 454 (1971): 903- 995
doi: 10.1098/rspa.1998.0193
17 GAUR P, KAUSHIK G, PACHORI B, et al. Comparison analysis: single and multichannel EMD-based filtering with application to BCI [C]// Machine Intelligence and Signal Analysis. Singapore: Springer, 2019.
18 SHARMA R, PACHORI B. Automated classification of focal and non-focal EEG signals based on bivariate empirical mode decomposition [M]// Biomedical Signal and Image Processing in Patient Care. Indore: IGI, 2017.
19 GAUR P, PACHORI B, HUI W, et al. An empirical mode decomposition based filtering method for classification of motor-imagery EEG signals for enhancing brain-computer interface [C]// The International Joint Conference on Neural Networks. Killarney: IEEE, 2015.
20 CORETTO P, LEPORE N, BRIEVA J, et al. Open access database of EEG signals recorded during imagined speech [C]// 12th International Symposium on Medical Information Processing and Analysis. Tandil: SPIE, 2017: 1016002.
21 LI G, WANG S, LI M, et al Towards real-life EEG applications: novel superporous hydrogel-based semi-dry EEG electrodes enabling automatically ‘charge–discharge’electrolyte[J]. Journal of Neural Engineering, 2021, 18 (4): 046016
doi: 10.1088/1741-2552/abeeab
22 CAO Y, OOSTENVELD R, ALDAY M, et al Are alpha and beta oscillations spatially dissociated over the cortex in context-driven spoken-word production?[J]. Psychophysiology, 2022, (6): e13999
23 ALYASSERI A, KHADER T, Al A, et al. EEG signal denoising using hybridizing method between wavelet transform with genetic algorithm [C]// Proceedings of the 11th National Technical Seminar on Unmanned System Technology. Singapore: Springer, 2021: 449-469.
24 ALBORZ R, ROBERT T, AURELIEN B, et al EEG classification of covert speech using regularized neural networks[J]. IEEE/ACM Transactions on Audio, Speech and Language Processing, 2017, 25 (12): 2292- 2300
doi: 10.1109/TASLP.2017.2758164
25 李明阳, 陈万忠, 张涛 基于DD-DWT和Log-Logistic参数回归的癫痫脑电自动识别方法[J]. 仪器仪表学报, 2017, 38 (6): 1368- 1377
LI Ming-yang, CHEN Wan-zhong, ZHANG Tao Automatic EEG recognition for epilepsy based on DD-DWT and Log-Logistic regression[J]. Chinese Journal of Scientific Instrument, 2017, 38 (6): 1368- 1377
doi: 10.3969/j.issn.0254-3087.2017.06.007
26 RAJASHEKHAR U, NEELAPPA D, RAJESH L EEG signal classification for brain–computer interface using discrete wavelet transform (DWT)[J]. International Journal of Intelligent Unmanned Systems, 2021, 10 (1): 181- 188
27 ALSALEH M. Toward an imagined speech-based brain computer interface using EEG signals [D]. Sheffield: University of Sheffield, 2019.
28 王楚涵. 基于融合特征和集成分类的在线EEG情感识别系统研究[D]. 天津: 天津理工大学, 2021.
WANG Chu-han. Research on online EEG emotion recognition system based on Fusion feature and ensemble classification [D]. Tianjin: Tianjin University of Technology, 2021.
29 ZHOU J, HUANG S, WANG M, et al Performance evaluation of hybrid GA–SVM and GWO–SVM models to predict earthquake-induced liquefaction potential of soil: a multi-dataset investigation[J]. Engineering with Computers, 2021, (2): 1- 19
30 曾靖翔, 张金喜, 曹丹丹, 等 利用kNN方法的沥青路面平整度智能检测[J]. 华南理工大学学报: 自然科学版, 2022, 50 (3): 50- 56
ZENG Jing-xiang, ZHANG Jin-xi, CAO Dan-dan, et al Intelligent detection of asphalt pavement flatness by kNN method[J]. Journal of South China University of Technology: Natural Science Edition, 2022, 50 (3): 50- 56
31 GARCIA S, VILLASENOR L, REYES A, et al. Tensor decomposition for imagined speech discrimination in EEG [C]// Mexican International Conference on Artificial Intelligence. Cham: Springer, 2018: 239-249.
32 COONEY C, FOLLI R, COYLE D. Optimizing layers improves CNN generalization and transfer learning for imagined speech decoding from EEG [C]// IEEE International Conference on Systems, Man and Cybernetics. Bari: IEEE, 2019: 1311-1316.
[1] 王雅静,王群,李博闻,刘志文,朴媛媛,遇涛. 基于脑电信号预发作数据段选取的癫痫发作预测[J]. 浙江大学学报(工学版), 2020, 54(11): 2258-2265.
[2] 项贻强, 郏亚坤. 基于小波总能量相对变化的拱桥吊杆损伤识别[J]. 浙江大学学报(工学版), 2017, 51(5): 870-878.