|
|
Recognition of multi-class motor imagery EEG signals based on convolutional neural network |
Jin-zhen LIU1,2( ),Fang-fang YE1,2,Hui XIONG1,2,*( ) |
1. School of Control Science and Engineering, TIANGONG University, Tianjin 300387, China 2. Tianjin Key Laboratory of Advanced Technology of Electrical Engineering and Energy, TIANGONG University, Tianjin 300387, China |
|
|
Abstract An automatic classification method of motor imagery (MI) signals based on deep learning was proposed, aiming at the traditional multi-class MI recognition method of electroencephalogram (EEG) signals that requires cumbersome preprocessing and feature extraction. In terms of sample representation, the multi-channel EEG signals were converted into a one-dimensional sequence signal for processing, which increased the number of samples while neglecting the influence of spatial information related to the channel position. And according to the characteristics of the input signal, the multi-layer one-dimensional convolutional neural network was used to learn time-frequency information in EEG signals under different motor imagery states, and automatically complete the feature extraction and classification. The proposed method was compared with a variety of methods on the public dataset and the classification of the actual collected data was completed. The proposed method was used to do end-to-end learning of EEG signals without prior knowledge. Experimental results show that the method can obtain higher multi-classification accuracy and reduce the impact of individual differences on classification. The proposed method is conducive to the development of MI-based brain-computer interface systems.
|
Received: 28 December 2020
Published: 05 November 2021
|
|
Fund: 国家自然科学基金资助项目(61871288);天津市自然科学基金资助项目(18JCYBJC90400,18JCQNJC84000) |
Corresponding Authors:
Hui XIONG
E-mail: liujinzhen@tiangong.edu.cn;xionghui@tiangong.edu.cn
|
基于卷积神经网络的多类运动想象脑电信号识别
针对传统多类运动想象(MI)脑电信号的识别方法须进行繁琐的预处理以及特征提取问题,提出基于深度学习的MI信号自动分类方法.在样本表示方面,提出将多通道脑电(EEG)信号转化为一维序列信号处理,在增加样本数量的同时又能够忽略与通道位置相关的空间信息的影响;根据输入信号的特点,采用多层一维卷积神经网络学习不同运动想象状态时脑电信号中的时频信息,自动完成特征提取和分类工作. 将所提出的方法在公共数据集上与多种方法进行比较,并完成对实际采集数据集的分类. 利用所提方法在不需要先验知识的条件下,对脑电信号进行端到端的学习. 结果表明该方法可以获得更高的多分类准确率以及降低个体差异对分类的影响. 所提出的方法有利于促进基于MI 的脑机接口系统的开发.
关键词:
运动想象,
脑电信号,
卷积神经网络,
个体差异,
脑机接口
|
|
[1] |
GRAIMANN B, ALLISON B, PFURTSCHELLER G. Brain-computer interface: a gentle introduction [M]. The frontiers collection. Berlin, Heidelberg: Springer, 2009: 1-27.
|
|
|
[2] |
ADAMS M, BEN-SALEM S, ISLAM Z, et al. Towards an SSVEP-BCI controlled smart home [C]// 2019 IEEE International Conference on Systems Man and Cybernetics (SMC). Bari: IEEE, 2019: 2737-2742.
|
|
|
[3] |
CHEN L, WANG Z P, HE F, et al. An online hybrid brain-computer interface combining multiple physiological signals for webpage browse [C]// 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Milan: IEEE, 2015: 1152-1155.
|
|
|
[4] |
WONG C M, TANG Q, WAN F, et al. A multi-channel SSVEP-based BCI for computer games with analogue control [C]// Proceedings of the 2015 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Application (CIVEMSA). Shenzhen: IEEE, 2015: 1-6.
|
|
|
[5] |
PFURTSCHELLER G, LOPES DA SILVA F H Event-related EEG/MEG synchronization and desynchronization: basic principles[J]. Clinical Neurophysiology, 1999, 110 (11): 1842- 1857
doi: 10.1016/S1388-2457(99)00141-8
|
|
|
[6] |
巫嘉陵, 高忠科 脑机接口技术及其在神经科学中的应用[J]. 中国现代神经疾病杂志, 2021, 21 (1): 3- 8 WU Jia-ling, GAO Zhong-ke Brain-computer interface technology and its applications in neuroscience[J]. Chinese Journal of Modern Neurological Diseases, 2021, 21 (1): 3- 8
doi: 10.3969/j.issn.1672-6731.2021.01.002
|
|
|
[7] |
BAIG M Z, ASLAML N, SHUM H P H Filtering techniques for channel selection in motor imagery EEG applications: a survey[J]. Artificial Intelligence Review, 2020, 53 (3): 1207- 1232
|
|
|
[8] |
霍首君, 郝琰, 石慧宇, 等 基于深度卷积网络的运动想象脑电信号模式识别[J]. 计算机应用, 2020, 41 (4): 1042- 1048 HUO Shou-jun, HAO Yan, SHI Hui-yu, et al Pattern recognition of motor imagery EEG based on deep convolutional networks[J]. Computer Applications, 2020, 41 (4): 1042- 1048
|
|
|
[9] |
CHATTERJEE R, BANDYOPADHYAY T, SANYAL D K, et al. Comparative analysis of feature extraction techniques in motor imagery EEG signal classification [C]// Proceedings of First International Conference on Smart System, Innovations and Computing. Smart Innovation, Systems and Technologies. Jaipur: Springer, 2018, 79: 73-83.
|
|
|
[10] |
WANG Y J, GAO S K, GAO X R, et al. Common spatial pattern method for channel selection in motor imagery based brain-computer interface [C]// International Conference of the Engineering in Medicine and Biology Society. Shanghai: IEEE, 2005: 5392-5395.
|
|
|
[11] |
ANG K K, CHINY C, ZHANG H H, et al. Filter bank common spatial pattern (FBCSP) in brain-computer interface [C]// Proceedings of the IEEE International Joint Conference on Neural Networks. Hong Kong: IEEE, 2008: 2391–2398.
|
|
|
[12] |
施锦河, 沈继忠, 王攀 四类运动想象脑电信号特征提取与分类算法[J]. 浙江大学学报:工学版, 2012, 46 (2): 338- 344 SHI Jin-he, SHEN Ji-zhong, WANG Pan Feature extraction and classification of four-class motor imagery EEG data[J]. Journal of Zhejiang University: Engineering Science, 2012, 46 (2): 338- 344
doi: 10.3785/j.issn.1008-973X.2012.02.025
|
|
|
[13] |
TYAGI A, NEHRA V. Time frequency analysis of non-stationary motor imagery EEG signal [C]// 2017 International Conference on Computing and Communication Technologies for Smart Nation (IC3TSN). Gurgaon: IEEE, 2017: 44-50.
|
|
|
[14] |
LIU J, CHENG Y, ZHANG W. Deep learning EEG response representation for brain computer interface [C]// 34th Chinese Control Conference. Hangzhou: IEEE, 2015: 3518-3523.
|
|
|
[15] |
KUMAR S, SHARMA A, MAMUN K, et al. A deep learning approach for motor imagery EEG signal classification [C]// 2016 3rd Asia-Pacific World Congress on Computer Science and Engineering (APWC on CSE). Nadi: IEEE, 2016: 34-39.
|
|
|
[16] |
LEE H K, CHOI Y. A convolutional neural networks scheme for classification of motor imagery EEG based on wavelet time-frequency image [C]// 2018 International Conference on Information Networking (ICOIN). Chiang Mai: IEEE, 2018: 906-909.
|
|
|
[17] |
ZHU X Y, LI P Y, LI C B, et al Separated channel convolutional neural network to realize the training free motor imagery BCI systems[J]. Biomedical Signal Processing and Control, 2019, 49: 396- 403
doi: 10.1016/j.bspc.2018.12.027
|
|
|
[18] |
YANG T, PHUA K S, YU J H, et al. Image-based motor imagery EEG classification using convolutional neural network [C]// 2019 IEEE EMBS International Conference on Biomedical and Health Informatics (BHI). Chicago: IEEE, 2019: 1-4.
|
|
|
[19] |
LI Y, ZHANG X R, LEI M Y, et al A channel-projection mixed-scale convolutional neural network for motor imagery EEG decoding[J]. IEEE Transaction on Neural Systems and Rehabilitation Engineering, 2019, 27 (6): 1170- 1180
doi: 10.1109/TNSRE.2019.2915621
|
|
|
[20] |
WU H, NIU Y, LI F, et al A parallel multiscale filter bank convolutional neural networks for motor imagery EEG classification[J]. Frontiers in Neuroscience, 2019, 26 (13): 1275
|
|
|
[21] |
ROY S, MCCREADIE K, PRASAD G. Can a single model deep learning approach enhance classification accuracy of an EEG-based brain-computer interface? [C]// 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC). Bari: IEEE, 2019: 1317-1321.
|
|
|
[22] |
TANG X B, ZHAO J C, FU W L, et al. A novel classification algorithm for MI-EEG based on deep learning [C]// 2019 IEEE 8th Joint International Information Technology and Artificial Intelligence Conference (ITAIC). Chongqing: IEEE, 2019: 606-611.
|
|
|
[23] |
SHEN Y R, LU H T, JIA J. Classification of motor imagery EEG signal with deep learning models [C]// International Conference on Intelligent Science and Big Data Engineering. Dalian: Springer, 2017: 181-190.
|
|
|
[24] |
LASHGARI E, LIANG D, MAOZ U Data augmentation for deep-learning-based electroencephalography[J]. Journal of Neuroscience Methods, 2020, 346: 108885
doi: 10.1016/j.jneumeth.2020.108885
|
|
|
[25] |
BASHIVAN P, RISH I, YEASIN M, et al. Learning representations from EEG with deep recurrent-convolutional neural networks [EB/OL]. (2015-11-19) [2020-12-28]. https://arxiv.org/abs/1511.06448v3.
|
|
|
[26] |
CHANG L, DENG X M, ZHOU M Q, et al Convolutional neural networks in image understanding[J]. Acta Automatica Sinica, 2016, 42 (9): 1300- 1312
|
|
|
[27] |
DING Y, ROBINSON N, ZENG Q H, et al. TSception: a deep learning framework for emotion detection using EEG [EB/OL]. (2020-04-02) [2020-12-28]. https://arxiv.org/abs/2004.02965.
|
|
|
[28] |
王卫星, 孙守迁, 李超, 等 基于卷积神经网络的脑电信号上肢运动意图识别[J]. 浙江大学学报:工学版, 2017, 51 (7): 1381- 1389 WANG Wei-xing, SUN Shou-qian, LI Chao, et al Recognition of upper limb motion intention of EEG signal based on convolutional neural network[J]. Journal of Zhejiang University: Engineering Science, 2017, 51 (7): 1381- 1389
|
|
|
[29] |
TANGERMANN M, MÜLLER K R, AERTSEN A, et al Review of the BCI competition IV[J]. Frontiers Neuroscience, 2012, 6: 55
|
|
|
[30] |
LI D L, WANG J H, XU J C, et al Densely feature fusion based on convolutional neural networks for motor imagery EEG classification[J]. IEEE Access, 2019, 7: 132720- 132730
doi: 10.1109/ACCESS.2019.2941867
|
|
|
[31] |
DAI G, ZHOU J, HUANG J, et al HS-CNN: a CNN with hybrid convolutional scale for EEG motor imagery classification[J]. Journal of Neural Engineering, 2020, 17 (1): 016025
doi: 10.1088/1741-2552/ab405f
|
|
|
[32] |
MAJIDOV I, WHANGBO T Efficient classification of motor imagery electroencephalography signal using deep learning methods[J]. Sensors, 2019, 19 (7): 1736- 1749
doi: 10.3390/s19071736
|
|
|
[33] |
ZHANG R, QUN Z, DOU L, et al A novel hybrid deep learning scheme for four-class motor imagery classification[J]. Journal of Neural Engineering, 2019, 16 (6): 1- 11
|
|
|
[34] |
MA X G, WANG D S, LIU D H, et al DWT and CNN based multi-class motor imagery electroencephalographic signal recognition[J]. Journal of Neural Engineering, 2020, 17 (1): 016073
doi: 10.1088/1741-2552/ab6f15
|
|
|
[35] |
GAUR P, PACHORI R B, WANG H, et al A multi-class EEG-based BCI classification using multivariate empirical mode decomposition based filtering and Riemannian Geometry[J]. Expert Systems with Applications, 2018, 95: 201- 211
doi: 10.1016/j.eswa.2017.11.007
|
|
|
[36] |
XU J, ZHENG H, WANG J, et al Recognition of EEG signal motor imagery intention based on deep multi-view feature learning[J]. Sensors, 2020, 20 (12): 3496
doi: 10.3390/s20123496
|
|
|
[37] |
XU M, YAO J, ZHANG Z, et al Learning EEG topographical representation for classification via convolutional neural network[J]. Pattern Recognition, 2020, 105: 107390
doi: 10.1016/j.patcog.2020.107390
|
|
|
[38] |
SAKHAVI S, GUAN C, YAN S Learning temporal information for brain-computer interface using convolutional neural networks[J]. IEEE Transaction on Neural Networks and Learning Systems, 2018, 29 (11): 5619- 5629
doi: 10.1109/TNNLS.2018.2789927
|
|
|
[39] |
ZHAO X, ZHANG H, ZHU G, et al A multi-branch 3D convolutional neural network for EEG-based motor imagery classification[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2019, 27 (10): 2164- 2177
doi: 10.1109/TNSRE.2019.2938295
|
|
|
[40] |
ANG K K, CHIN Z Y, WANG C, et al Filter bank common spatial pattern algorithm on BCI Competition IV datasets 2a and 2b[J]. Frontiers in Neuroscience, 2012, 6: 39
|
|
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
|
Shared |
|
|
|
|
|
Discussed |
|
|
|
|