基于双分支网络的表面肌电信号识别方法

doi:10.3785/j.issn.1008-973X.2024.11.002

浙江大学学报(工学版)

2024, Vol. 58

Issue (11): 2208-2218 DOI: 10.3785/j.issn.1008-973X.2024.11.002

计算机技术、控制工程

基于双分支网络的表面肌电信号识别方法

王万良(

),潘杰,王铮,潘家宇

浙江工业大学计算机科学与技术学院，浙江杭州 310023

Recognition method of surface electromyographic signal based on two-branch network

Wanliang WANG(

),Jie PAN,Zheng WANG,Jiayu PAN

College of Computer Science and Technology, Zhejiang University of Technology, Hangzhou 310023, China

全文: PDF(2511 KB) HTML

摘要：

针对目前表面肌电信号(sEMG)手势识别细节信息提取不充分，对相似手势区分困难的问题，提出基于加强二维化特征的双分支网络(ETDTBN)模型. 该模型通过加强二维化方法生成二维特征图，使用多层卷积神经网络(ML-CNN)提取sEMG的空间特征，利用双向门控循环单元(Bi-GRU)提取原始信号的时序特征. 考虑到不同的特征对网络的影响程度不同，引入自适应特征融合机制对不同分支进行融合，强化有用特征并弱化无用特征，提高表面肌电识别的准确率. 实验在电极偏移和不同受试者2种情况下对ETDTBN进行训练与测试，与主流的肌电手势识别模型进行对比. 可知，ETDTBN的总体识别准确率分别为86.95%和84.15%，准确率均为最优，证明了该模型的有效性.

关键词： 表面肌电信号（sEMG）; 手势识别; 加强二维化特征; 双分支网络; 自适应特征融合机制

Abstract:

An enhanced two-dimensional feature based two-branch network (ETDTBN) was proposed aiming at the problems of insufficient detailed information extraction and difficulty in distinguishing similar gestures in surface electromyogram (sEMG) gesture recognition. Discrete features were converted into two-dimensional feature maps by the proposed enhanced two-dimensional method. Then a multi-layer convolutional neural network (ML-CNN) was used to extract the spatial features, while a bidirectional gated recurrent unit (Bi-GRU) was used to extract the temporal features from the original signal. A self-adaptive feature fusion mechanism was introduced to fuse different branches, strengthen useful features and weaken useless features in order to improve the accuracy of sEMG recognition by considering that different features had different degrees of influence on the network. Experiments were used to train and test the ETDTBN in two scenarios of electrode displacement and different subjects comparing with mainstream sEMG gesture recognition models. Results showed that the overall recognition accuracy of ETDTBN were 86.95% and 84.15%, respectively. Both accuracies are optimal, proving the effectiveness of the model.

Key words: surface electromyogram (sEMG) gesture recognition enhanced two-dimensional feature two-branch network self-adaptive feature fusion mechanism

收稿日期: 2023-09-26 出版日期: 2024-10-23

CLC:

TP 393

基金资助: 国家自然科学基金资助项目(51875524, 61873240)；浙江大学CAD&CG国家重点实验室开放课题资助项目(A2210).

作者简介: 王万良（1957—），男，教授，从事人工智能及其自动化、网络控制的研究. orcid.org/0000-0002-1552-5075. E-mail： zjutwwl@zjut.edu.cn

	服务
	把本文推荐给朋友
	加入引用管理器
	E-mail Alert
	作者相关文章
	王万良
	潘杰
	王铮
	潘家宇

引用本文:

王万良,潘杰,王铮,潘家宇. 基于双分支网络的表面肌电信号识别方法[J]. 浙江大学学报(工学版), 2024, 58(11): 2208-2218.

Wanliang WANG,Jie PAN,Zheng WANG,Jiayu PAN. Recognition method of surface electromyographic signal based on two-branch network. Journal of ZheJiang University (Engineering Science), 2024, 58(11): 2208-2218.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2024.11.002 或 https://www.zjujournals.com/eng/CN/Y2024/V58/I11/2208

图 1 ETDTBN的整体结构

图 2 双分支网络的实现细节

图 3 Bi-GRU的展开结构

图 4 自适应特征融合结构

图 5 手势动作的示意图

图 6 窗口分析法

图 7 ETDTBN网络的结构

表 1 ETDTBN网络的结构参数

表 2 电极偏移情况下各方法的识别效果

图 8 电极偏移情况下各模型的分类准确率

图 9 电极偏移情况下不同模型的混淆矩阵比较

表 3 不同受试者情况下各方法的识别效果

图 10 不同受试者情况下各模型的分类准确率变化

图 11 不同受试者情况下不同模型的混淆矩阵

表 4 融合策略对表面肌电信号分类准确率的影响

表 5 所有方法的计算资源分析

表 6 各方法在公开数据集上的识别效果

1	BHAUMIK G, VERMA M, GOVIL M C, et al Hyfinet: hybrid feature attention network for hand gesture recognition[J]. Multimedia Tools and Applications, 2023, 82 (4): 4863- 4882 doi: 10.1007/s11042-021-11623-3
2	都明宇, 鲍官军, 杨庆华, 等基于改进支持向量机的人手动作模式识别方法[J]. 浙江大学学报: 工学版, 2018, 52 (7): 1239- 1246 DOU Mingyu, BAO Guanjun, YANG Qinghua, et al Novel method in pattern recognition of hand actions based on improved support vector machine[J]. Journal of Zhejiang University: Engineering Science, 2018, 52 (7): 1239- 1246
3	赵翠莲, 徐浩宇, 罗林辉, 等熵在不同等级偏瘫患者sEMG运动检测中的应用[J]. 浙江大学学报: 工学版, 2018, 52 (4): 798- 805 ZHAO Cuilian, XU Haoyu, LUO Linhui, et al SEMG activity detection of hemiplegic patients in different stages using entropy algorithms[J]. Journal of Zhejiang University: Engineering Science, 2018, 52 (4): 798- 805
4	CHENG Y, LI G, YU M, et al Gesture recognition based on surface electromyography-feature image[J]. Concurrency and Computation: Practice and Experience, 2021, 33 (6): e6051 doi: 10.1002/cpe.6051
5	SUK H I, SIN B K, LEE S W Hand gesture recognition based on dynamic Bayesian network framework[J]. Pattern Recognition, 2010, 43 (9): 3059- 3072 doi: 10.1016/j.patcog.2010.03.016
6	QI J, JIANG G, LI G, et al Intelligent human-computer interaction based on surface EMG gesture recognition[J]. IEEE Access, 2019, 7: 61378- 61387 doi: 10.1109/ACCESS.2019.2914728
7	BARGELLESI N, CARLETTI M, CENEDESE A, et al A random forest-based approach for hand gesture recognition with wireless wearable motion capture sensors[J]. IFAC-PapersOnLine, 2019, 52 (11): 128- 133 doi: 10.1016/j.ifacol.2019.09.129
8	CHEN H, ZHANG Y, LI G, et al Surface electromyography feature extraction via convolutional neural network[J]. International Journal of Machine Learning and Cybernetics, 2020, 11 (1): 185- 196 doi: 10.1007/s13042-019-00966-x
9	WANG H, ZHANG Y, LIU C, et al sEMG based hand gesture recognition with deformable convolutional network[J]. International Journal of Machine Learning and Cybernetics, 2022, 13 (6): 1729- 1738 doi: 10.1007/s13042-021-01482-7
10	HU Y, WONG Y, WEI W, et al A novel attention-based hybrid CNN-RNN architecture for sEMG-based gesture recognition[J]. PloS one, 2018, 13 (10): e0206049 doi: 10.1371/journal.pone.0206049
11	TSINGANOS P, CORNELIS B, CORNELIS J, et al. Improved gesture recognition based on sEMG signals and TCN [C]// IEEE International Conference on Acoustics, Speech and Signal Processing . Brighton: IEEE, 2019: 1169-1173.
12	WANG W, YOU W, WANG Z, et al Feature fusion-based improved capsule network for sEMG signal recognition[J]. Computational Intelligence and Neuroscience, 2022, (1): 7603319
13	KUMAR V, MINZ S Feature selection: a literature review[J]. SmartCR, 2014, 4 (3): 211- 229
14	CHEN G, WANG W, WANG Z, et al Two-dimensional discrete feature based spatial attention CapsNet for sEMG signal recognition[J]. Applied Intelligence, 2020, 50: 3503- 3520 doi: 10.1007/s10489-020-01725-0
15	GU J, WANG Z, KUEN J, et al Recent advances in convolutional neural networks[J]. Pattern Recognition, 2018, 77: 354- 377 doi: 10.1016/j.patcog.2017.10.013
16	KRIZHEVSKY A, SUTSKEVER I, HINTON G E ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60 (6): 84- 90 doi: 10.1145/3065386
17	O'SHEA K, NASH R. An introduction to convolutional neural networks [EB/OL]. [2023-09-01]. https://arxiv.org/abs/1511.08458.
18	SHERSTINSKY A Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network[J]. Physica D: Nonlinear Phenomena, 2020, 404: 132306 doi: 10.1016/j.physd.2019.132306
19	YU Y, SI X, HU C, et al A review of recurrent neural networks: LSTM cells and network architectures[J]. Neural Computation, 2019, 31 (7): 1235- 1270 doi: 10.1162/neco_a_01199
20	SHEWALKAR A, NYAVANANDI D, LUDWIG S A Performance evaluation of deep neural networks applied to speech recognition: RNN, LSTM and GRU[J]. Journal of Artificial Intelligence and Soft Computing Research, 2019, 9 (4): 235- 245 doi: 10.2478/jaiscr-2019-0006
21	CHO K, VAN M B, GULCEHRE C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation [EB/OL]. [2023-09-01]. https://arxiv.org/abs/1406.1078.
22	BERGLUND M, RAIKO T, HONKALA M, et al. Bidirectional recurrent neural networks as generative models [EB/OL]. [2023-09-01]. https://proceedings.neurips.cc/paper/2015/hash/c75b6f114c23a4d7ea11331e7c00e73c-Abstract.html.
23	LI X, WANG W, HU X, et al. Selective kernel networks [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Long Beach: IEEE, 2019: 510-519.
24	ATZORI M, GIJSBERTS A, HEYNEN S, et al. Building the Ninapro database: a resource for the biorobotics community [C]// 4th IEEE RAS and EMBS International Conference on Biomedical Robotics and Biomechatronics . Rome: IEEE, 2012: 1258-1265.
25	NAZMI N, RAHMAN M A, YAMAMOTO S, et al Assessment on stationarity of EMG signals with different windows size during isotonic contractions[J]. Applied Sciences, 2017, 7 (10): 1050 doi: 10.3390/app7101050
26	HUDGINS B, PARKER P, SCOTT R N A new strategy for multifunction myoelectric control[J]. IEEE Transactions on Biomedical Engineering, 1993, 40 (1): 82- 94 doi: 10.1109/10.204774
27	PHINYOMARK A, PHUKPATTARANONT P, LIMSAKUL C Feature reduction and selection for EMG signal classification[J]. Expert Systems with Applications, 2012, 39 (8): 7420- 7431 doi: 10.1016/j.eswa.2012.01.102
28	ABBASPOUR S, LINDEN M, GHOLAMHOSSEINI H, et al Evaluation of surface EMG-based recognition algorithms for decoding hand movements[J]. Medical and Biological Engineering and Computing, 2020, 58: 83- 100 doi: 10.1007/s11517-019-02073-z
29	ZI L I, ZA X G, BI Z, et al Review of sEMG-based motion intent recognition methods in non-ideal conditions[J]. Acta Automatica Sinica, 2021, 47 (5): 955- 969
30	PAUL Y, GOYAL V, JASWAL R A. Comparative analysis between SVM & KNN classifier for EMG signal classification on elementary time domain features [C]// 4th International Conference on Signal Processing, Computing and Control . Solan: IEEE, 2017: 169-175.
31	NARAYAN Y Comparative analysis of SVM and Naive Bayes classifier for the SEMG signal classification[J]. Materials Today: Proceedings, 2021, 37: 3241- 3245 doi: 10.1016/j.matpr.2020.09.093
32	HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . [S. l. ]: IEEE, 2016: 770-778.
33	ZHAO R, WANG D, YAN R, et al Machine health monitoring using local feature-based gated recurrent unit networks[J]. IEEE Transactions on Industrial Electronics, 2017, 65 (2): 1539- 1548
34	GRAVES A, SCHMIDHUBER J Framewise phoneme classification with bidirectional LSTM and other neural network architectures[J]. Neural Networks, 2005, 18 (5/6): 602- 610
35	SHEN S, WANG X, MAO F, et al Movements classification through sEMG with convolutional vision transformer and stacking ensemble learning[J]. IEEE Sensors Journal, 2022, 22 (13): 13318- 13325 doi: 10.1109/JSEN.2022.3179535
36	WU Y, ZHENG B, ZHAO Y. Dynamic gesture recognition based on LSTM-CNN [C]// Chinese Automation Congress . Xi’an: IEEE, 2018: 2446-2450.

[1]	周雕,熊馨,周建华,宗静,张琪. 卷积神经网络结合子域适应的低采样率肌电手势识别[J]. 浙江大学学报(工学版), 2024, 58(10): 2011-2019.
[2]	李卓峰,孙铭会. 基于高采样率惯性测量单元的手部状态与手势识别[J]. 浙江大学学报(工学版), 2023, 57(3): 503-511.
[3]	辛文斌,郝惠敏,卜明龙,兰媛,黄家海,熊晓燕. 基于ShuffleNetv2-YOLOv3模型的静态手势实时识别方法[J]. 浙江大学学报(工学版), 2021, 55(10): 1815-1824.
[4]	都明宇, 鲍官军, 杨庆华, 王志恒, 张立彬. 基于改进支持向量机的人手动作模式识别方法[J]. 浙江大学学报(工学版), 2018, 52(7): 1239-1246.
[5]	赵翠莲, 徐浩宇, 罗林辉, 王凯. 熵在不同等级偏瘫患者sEMG运动检测中的应用[J]. 浙江大学学报(工学版), 2018, 52(4): 798-805.
[6]	刘磊, 杨鹏, 刘作军. 采用多核相关向量机的人体步态识别[J]. 浙江大学学报(工学版), 2017, 51(3): 562-571.
[7]	跃龙,陈岭,陈根才. 基于层次化BoF模型和Spectral-HIK过滤的手势识别算法闯[J]. J4, 2013, 47(9): 1531-1536.

Viewed

Full text

Abstract

Cited

Shared

Discussed