|
|
|
| Recognition method of surface electromyographic signal based on two-branch network |
Wanliang WANG( ),Jie PAN,Zheng WANG,Jiayu PAN |
| College of Computer Science and Technology, Zhejiang University of Technology, Hangzhou 310023, China |
|
|
|
Abstract An enhanced two-dimensional feature based two-branch network (ETDTBN) was proposed aiming at the problems of insufficient detailed information extraction and difficulty in distinguishing similar gestures in surface electromyogram (sEMG) gesture recognition. Discrete features were converted into two-dimensional feature maps by the proposed enhanced two-dimensional method. Then a multi-layer convolutional neural network (ML-CNN) was used to extract the spatial features, while a bidirectional gated recurrent unit (Bi-GRU) was used to extract the temporal features from the original signal. A self-adaptive feature fusion mechanism was introduced to fuse different branches, strengthen useful features and weaken useless features in order to improve the accuracy of sEMG recognition by considering that different features had different degrees of influence on the network. Experiments were used to train and test the ETDTBN in two scenarios of electrode displacement and different subjects comparing with mainstream sEMG gesture recognition models. Results showed that the overall recognition accuracy of ETDTBN were 86.95% and 84.15%, respectively. Both accuracies are optimal, proving the effectiveness of the model.
|
|
Received: 26 September 2023
Published: 23 October 2024
|
|
|
| Fund: 国家自然科学基金资助项目(51875524, 61873240)；浙江大学CAD&CG国家重点实验室开放课题资助项目(A2210). |
基于双分支网络的表面肌电信号识别方法
针对目前表面肌电信号(sEMG)手势识别细节信息提取不充分，对相似手势区分困难的问题，提出基于加强二维化特征的双分支网络(ETDTBN)模型. 该模型通过加强二维化方法生成二维特征图，使用多层卷积神经网络(ML-CNN)提取sEMG的空间特征，利用双向门控循环单元(Bi-GRU)提取原始信号的时序特征. 考虑到不同的特征对网络的影响程度不同，引入自适应特征融合机制对不同分支进行融合，强化有用特征并弱化无用特征，提高表面肌电识别的准确率. 实验在电极偏移和不同受试者2种情况下对ETDTBN进行训练与测试，与主流的肌电手势识别模型进行对比. 可知，ETDTBN的总体识别准确率分别为86.95%和84.15%，准确率均为最优，证明了该模型的有效性.
关键词：
表面肌电信号（sEMG）,
手势识别,
加强二维化特征,
双分支网络,
自适应特征融合机制
|
|
| [1] |
BHAUMIK G, VERMA M, GOVIL M C, et al Hyfinet: hybrid feature attention network for hand gesture recognition[J]. Multimedia Tools and Applications, 2023, 82 (4): 4863- 4882
doi: 10.1007/s11042-021-11623-3
|
|
|
| [2] |
都明宇, 鲍官军, 杨庆华, 等 基于改进支持向量机的人手动作模式识别方法[J]. 浙江大学学报: 工学版, 2018, 52 (7): 1239- 1246
DOU Mingyu, BAO Guanjun, YANG Qinghua, et al Novel method in pattern recognition of hand actions based on improved support vector machine[J]. Journal of Zhejiang University: Engineering Science, 2018, 52 (7): 1239- 1246
|
|
|
| [3] |
赵翠莲, 徐浩宇, 罗林辉, 等 熵在不同等级偏瘫患者sEMG运动检测中的应用[J]. 浙江大学学报: 工学版, 2018, 52 (4): 798- 805
ZHAO Cuilian, XU Haoyu, LUO Linhui, et al SEMG activity detection of hemiplegic patients in different stages using entropy algorithms[J]. Journal of Zhejiang University: Engineering Science, 2018, 52 (4): 798- 805
|
|
|
| [4] |
CHENG Y, LI G, YU M, et al Gesture recognition based on surface electromyography-feature image[J]. Concurrency and Computation: Practice and Experience, 2021, 33 (6): e6051
doi: 10.1002/cpe.6051
|
|
|
| [5] |
SUK H I, SIN B K, LEE S W Hand gesture recognition based on dynamic Bayesian network framework[J]. Pattern Recognition, 2010, 43 (9): 3059- 3072
doi: 10.1016/j.patcog.2010.03.016
|
|
|
| [6] |
QI J, JIANG G, LI G, et al Intelligent human-computer interaction based on surface EMG gesture recognition[J]. IEEE Access, 2019, 7: 61378- 61387
doi: 10.1109/ACCESS.2019.2914728
|
|
|
| [7] |
BARGELLESI N, CARLETTI M, CENEDESE A, et al A random forest-based approach for hand gesture recognition with wireless wearable motion capture sensors[J]. IFAC-PapersOnLine, 2019, 52 (11): 128- 133
doi: 10.1016/j.ifacol.2019.09.129
|
|
|
| [8] |
CHEN H, ZHANG Y, LI G, et al Surface electromyography feature extraction via convolutional neural network[J]. International Journal of Machine Learning and Cybernetics, 2020, 11 (1): 185- 196
doi: 10.1007/s13042-019-00966-x
|
|
|
| [9] |
WANG H, ZHANG Y, LIU C, et al sEMG based hand gesture recognition with deformable convolutional network[J]. International Journal of Machine Learning and Cybernetics, 2022, 13 (6): 1729- 1738
doi: 10.1007/s13042-021-01482-7
|
|
|
| [10] |
HU Y, WONG Y, WEI W, et al A novel attention-based hybrid CNN-RNN architecture for sEMG-based gesture recognition[J]. PloS one, 2018, 13 (10): e0206049
doi: 10.1371/journal.pone.0206049
|
|
|
| [11] |
TSINGANOS P, CORNELIS B, CORNELIS J, et al. Improved gesture recognition based on sEMG signals and TCN [C]// IEEE International Conference on Acoustics, Speech and Signal Processing . Brighton: IEEE, 2019: 1169-1173.
|
|
|
| [12] |
WANG W, YOU W, WANG Z, et al Feature fusion-based improved capsule network for sEMG signal recognition[J]. Computational Intelligence and Neuroscience, 2022, (1): 7603319
|
|
|
| [13] |
KUMAR V, MINZ S Feature selection: a literature review[J]. SmartCR, 2014, 4 (3): 211- 229
|
|
|
| [14] |
CHEN G, WANG W, WANG Z, et al Two-dimensional discrete feature based spatial attention CapsNet for sEMG signal recognition[J]. Applied Intelligence, 2020, 50: 3503- 3520
doi: 10.1007/s10489-020-01725-0
|
|
|
| [15] |
GU J, WANG Z, KUEN J, et al Recent advances in convolutional neural networks[J]. Pattern Recognition, 2018, 77: 354- 377
doi: 10.1016/j.patcog.2017.10.013
|
|
|
| [16] |
KRIZHEVSKY A, SUTSKEVER I, HINTON G E ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60 (6): 84- 90
doi: 10.1145/3065386
|
|
|
| [17] |
O'SHEA K, NASH R. An introduction to convolutional neural networks [EB/OL]. [2023-09-01]. https://arxiv.org/abs/1511.08458.
|
|
|
| [18] |
SHERSTINSKY A Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network[J]. Physica D: Nonlinear Phenomena, 2020, 404: 132306
doi: 10.1016/j.physd.2019.132306
|
|
|
| [19] |
YU Y, SI X, HU C, et al A review of recurrent neural networks: LSTM cells and network architectures[J]. Neural Computation, 2019, 31 (7): 1235- 1270
doi: 10.1162/neco_a_01199
|
|
|
| [20] |
SHEWALKAR A, NYAVANANDI D, LUDWIG S A Performance evaluation of deep neural networks applied to speech recognition: RNN, LSTM and GRU[J]. Journal of Artificial Intelligence and Soft Computing Research, 2019, 9 (4): 235- 245
doi: 10.2478/jaiscr-2019-0006
|
|
|
| [21] |
CHO K, VAN M B, GULCEHRE C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation [EB/OL]. [2023-09-01]. https://arxiv.org/abs/1406.1078.
|
|
|
| [22] |
BERGLUND M, RAIKO T, HONKALA M, et al. Bidirectional recurrent neural networks as generative models [EB/OL]. [2023-09-01]. https://proceedings.neurips.cc/paper/2015/hash/c75b6f114c23a4d7ea11331e7c00e73c-Abstract.html.
|
|
|
| [23] |
LI X, WANG W, HU X, et al. Selective kernel networks [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Long Beach: IEEE, 2019: 510-519.
|
|
|
| [24] |
ATZORI M, GIJSBERTS A, HEYNEN S, et al. Building the Ninapro database: a resource for the biorobotics community [C]// 4th IEEE RAS and EMBS International Conference on Biomedical Robotics and Biomechatronics . Rome: IEEE, 2012: 1258-1265.
|
|
|
| [25] |
NAZMI N, RAHMAN M A, YAMAMOTO S, et al Assessment on stationarity of EMG signals with different windows size during isotonic contractions[J]. Applied Sciences, 2017, 7 (10): 1050
doi: 10.3390/app7101050
|
|
|
| [26] |
HUDGINS B, PARKER P, SCOTT R N A new strategy for multifunction myoelectric control[J]. IEEE Transactions on Biomedical Engineering, 1993, 40 (1): 82- 94
doi: 10.1109/10.204774
|
|
|
| [27] |
PHINYOMARK A, PHUKPATTARANONT P, LIMSAKUL C Feature reduction and selection for EMG signal classification[J]. Expert Systems with Applications, 2012, 39 (8): 7420- 7431
doi: 10.1016/j.eswa.2012.01.102
|
|
|
| [28] |
ABBASPOUR S, LINDEN M, GHOLAMHOSSEINI H, et al Evaluation of surface EMG-based recognition algorithms for decoding hand movements[J]. Medical and Biological Engineering and Computing, 2020, 58: 83- 100
doi: 10.1007/s11517-019-02073-z
|
|
|
| [29] |
ZI L I, ZA X G, BI Z, et al Review of sEMG-based motion intent recognition methods in non-ideal conditions[J]. Acta Automatica Sinica, 2021, 47 (5): 955- 969
|
|
|
| [30] |
PAUL Y, GOYAL V, JASWAL R A. Comparative analysis between SVM & KNN classifier for EMG signal classification on elementary time domain features [C]// 4th International Conference on Signal Processing, Computing and Control . Solan: IEEE, 2017: 169-175.
|
|
|
| [31] |
NARAYAN Y Comparative analysis of SVM and Naive Bayes classifier for the SEMG signal classification[J]. Materials Today: Proceedings, 2021, 37: 3241- 3245
doi: 10.1016/j.matpr.2020.09.093
|
|
|
| [32] |
HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . [S. l. ]: IEEE, 2016: 770-778.
|
|
|
| [33] |
ZHAO R, WANG D, YAN R, et al Machine health monitoring using local feature-based gated recurrent unit networks[J]. IEEE Transactions on Industrial Electronics, 2017, 65 (2): 1539- 1548
|
|
|
| [34] |
GRAVES A, SCHMIDHUBER J Framewise phoneme classification with bidirectional LSTM and other neural network architectures[J]. Neural Networks, 2005, 18 (5/6): 602- 610
|
|
|
| [35] |
SHEN S, WANG X, MAO F, et al Movements classification through sEMG with convolutional vision transformer and stacking ensemble learning[J]. IEEE Sensors Journal, 2022, 22 (13): 13318- 13325
doi: 10.1109/JSEN.2022.3179535
|
|
|
| [36] |
WU Y, ZHENG B, ZHAO Y. Dynamic gesture recognition based on LSTM-CNN [C]// Chinese Automation Congress . Xi’an: IEEE, 2018: 2446-2450.
|
|
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
| |
Shared |
|
|
|
|
| |
Discussed |
|
|
|
|
