卷积神经网络结合子域适应的低采样率肌电手势识别

doi:10.3785/j.issn.1008-973X.2024.10.004

浙江大学学报(工学版)

2024, Vol. 58

Issue (10): 2011-2019 DOI: 10.3785/j.issn.1008-973X.2024.10.004

计算机与控制工程

卷积神经网络结合子域适应的低采样率肌电手势识别

周雕(

),熊馨,周建华*(

),宗静,张琪

昆明理工大学信息工程与自动化学院，云南昆明 650500

Convolutional neural network combined with subdomain adaptation for low sampling rate EMG-based gesture recognition

Diao ZHOU(

),Xin XIONG,Jianhua ZHOU*(

),Jing ZONG,Qi ZHANG

Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, China

全文: PDF(1146 KB) HTML

摘要：

为了提升模型识别低采样率肌电手势的性能，提出新的识别方法. 通过信息扩展层对预处理后的低采样率肌电信号信息进行扩展，增强关键特征的表示. 在特征提取网络中，利用子域适应网络提取源域与目标域中的域不变特征后进行域不变特征分类. 使用NinaPro数据库中的DB1和DB5子数据库对所提方法进行评估. 实验结果表明，所提方法识别53种和52种手势的最高准确率分别为90.89%(DB1)、89.90%(DB5)和82.01%(DB1)、77.07%(DB5)，能够降低电极移位、肌肉疲劳、皮肤阻抗的变化和肌肉相对电极的相对运动等因素对低采样率肌电手势识别的影响.

关键词： 低采样率表面肌电; 手势识别; 子域适应; 信息扩展; 挤压与激励注意力机制

Abstract:

A new recognition method was proposed to improve the performance of low sampling rate electromyography (EMG)-based gesture recognition. The information of the pre-processed low sampling rate EMG signal was extended by an information extension layer, and the representation of key features was enhanced. In the feature extraction network, domain invariant features in the source and target domains were extracted by the subdomain adaptation network, then the domain invariant features were classified. The proposed method was evaluated using the DB1 and DB5 sub-databases of the NinaPro database. Experimental results showed that the proposed method recognized 53 and 52 gestures with the highest accuracy of 90.89% (DB1), 89.90% (DB5) and 82.01% (DB1), 77.07% (DB5), respectively. The effects of factors on low sampling rate EMG-based gesture recognition are reduced by the proposed method, factors that include electrode shift, muscle fatigue, changes in skin impedance, and the relative movement of the muscle relative to the electrodes.

Key words: low sampling rate surface electromyography gesture recognition subdomain adaptation information expansion squeeze-and-excitation attention mechanism

收稿日期: 2023-09-12 出版日期: 2024-09-27

CLC:

TP 391

基金资助: 国家自然科学基金资助项目（82060329）.

通讯作者: 周建华 E-mail: 1962589274@qq.com;742028837@qq.com

作者简介: 周雕（1998—），男，硕士生，从事肌电信号处理与模式识别研究. orcid.org/0009-0004-8629-3509. E-mail：1962589274@qq.com

	服务
	把本文推荐给朋友
	加入引用管理器
	E-mail Alert
	作者相关文章
	周雕
	熊馨
	周建华
	宗静
	张琪

引用本文:

周雕,熊馨,周建华,宗静,张琪. 卷积神经网络结合子域适应的低采样率肌电手势识别[J]. 浙江大学学报(工学版), 2024, 58(10): 2011-2019.

Diao ZHOU,Xin XIONG,Jianhua ZHOU,Jing ZONG,Qi ZHANG. Convolutional neural network combined with subdomain adaptation for low sampling rate EMG-based gesture recognition. Journal of ZheJiang University (Engineering Science), 2024, 58(10): 2011-2019.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2024.10.004 或 https://www.zjujournals.com/eng/CN/Y2024/V58/I10/2011

表 1 NinaPro中DB1和DB5子数据库的信息

图 1 低采样率表面肌电手势识别模型的结构

表 2 卷积神经网络的参数

图 2 低采样率表面肌电手势识别模型中信息扩展层的结构

表 3 信息扩展层中各卷积层的参数

图 3 信息扩展层中挤压与激励注意力机制的结构

图 4 子域适应网络的结构

图 5 超参数实验的结果图

表 4 不同权重参数下53种手势的识别准确率

图 6 53种手势分类结果的混淆矩阵

表 5 不同权重参数下52种手势的识别准确率

图 7 52种手势分类结果的混淆矩阵

图 8 源域与目标域数据分布及2种模型的特征分布

表 6 不同类型手势动作的识别准确率

表 7 不同手势识别方法的性能比较

表 8 模型结构性能的消融实验

1	LI K, ZHANG J, WANG L, et al A review of the key technologies for sEMG-based human-robot interaction systems[J]. Biomedical Signal Processing and Control, 2020, 62: 102074 doi: 10.1016/j.bspc.2020.102074
2	JIANG S, KANG P, SONG X, et al Emerging wearable interfaces and algorithms for hand gesture recognition: a survey[J]. IEEE Reviews in Biomedical Engineering, 2021, 15: 85- 102
3	NGUYEN-TRONG K, VU H N, TRUNG N N, et al Gesture recognition using wearable sensors with bi-long short-term memory convolutional neural networks[J]. IEEE Sensors Journal, 2021, 21 (13): 15065- 15079 doi: 10.1109/JSEN.2021.3074642
4	JIANG X, XU K, LIU X, et al Neuromuscular password-based user authentication[J]. IEEE Transactions on Industrial Informatics, 2021, 17 (4): 2641- 2652 doi: 10.1109/TII.2020.3001612
5	CHEN X, GONG L, ZHENG L, et al. Soft exoskeleton glove for hand assistance based on human-machine interaction and machine learning [C]// 2020 IEEE International Conference on Human-Machine Systems . Rome: IEEE, 2020: 1−6.
6	Al-FAHAAM H, DAVIS S, NEFTI-MEZIANI S, et al Novel soft bending actuator-based power augmentation hand exoskeleton controlled by human intention[J]. Intelligent Service Robotics, 2018, 11 (3): 247- 268 doi: 10.1007/s11370-018-0250-4
7	PHINYOMARK A, KHUSHABA R N, SCHEME E Feature extraction and selection for myoelectric control based on wearable EMG sensors[J]. Sensors, 2018, 18 (5): 1615 doi: 10.3390/s18051615
8	CLANCT E A, MORIN E L, MERLETTI R Sampling, noise-reduction and amplitude estimation issues in surface electromyography[J]. Journal of Electromyography and Kinesiology, 2002, 12 (1): 1- 16 doi: 10.1016/S1050-6411(01)00033-5
9	IVES J C, WIGGLESWORTH J K Sampling rate effects on surface EMG timing and amplitude measures[J]. Clinical Biomechanics, 2003, 18 (6): 543- 552 doi: 10.1016/S0268-0033(03)00089-5
10	LI G, LI Y, YU L, et al Conditioning and sampling issues of EMG signals in motion recognition of multifunctional myoelectric prostheses[J]. Annals of Biomedical Engineering, 2011, 39 (6): 1779- 1787 doi: 10.1007/s10439-011-0265-x
11	PHINYOMARK A, SCHEME E. A feature extraction issue for myoelectric control based on wearable EMG sensors [C]// 2018 IEEE Sensors Applications Symposium . Seoul: IEEE, 2018: 1−6.
12	CHEN H, ZHANG Y, LI G, et al Surface electromyography feature extraction via convolutional neural network[J]. International Journal of Machine Learning and Cybernetics, 2020, 11: 185- 196 doi: 10.1007/s13042-019-00966-x
13	ATZORI M, GIJSBERTS A, HEYNEN S, et al. Building the Ninapro database: a resource for the biorobotics community [C]// 2012 4th IEEE RAS and EMBS International Conference on Biomedical Robotics and Biomechatronics . Rome: IEEE, 2012: 1258−1265.
14	CAO L, ZHANG W, KAN X, et al A novel adaptive mutation PSO optimized SVM algorithm for sEMG-based gesture recognition[J]. Scientific Programming, 2021, 2021 (1): 9988823
15	WAHID M F, TAFRESHI R, AL-SOWAIDI M, et al Subject-independent hand gesture recognition using normalization and machine learning algorithms[J]. Journal of Computational Science, 2018, 27: 69- 76 doi: 10.1016/j.jocs.2018.04.019
16	TEPE C, ERDIM M Classification of surface electromyography and gyroscopic signals of finger gestures acquired by Myo armband using machine learning methods[J]. Biomedical Signal Processing and Control, 2022, 75: 103588 doi: 10.1016/j.bspc.2022.103588
17	JARAMILLO-YANEZ A, UNAPANTA L, BENALCÁZAR M E. Short-term hand gesture recognition using electromyography in the transient state, support vector machines, and discrete wavelet transform [C]// 2019 IEEE Latin American Conference on Computational Intelligence . Guayaquil: IEEE, 2019: 1−6.
18	WAHID M F, TAFRESHI R, AL-SOWAIDI M, et al. An efficient approach to recognize hand gestures using machine-learning algorithms [C]// 2018 IEEE 4th Middle East Conference on Biomedical Engineering . Tunis: IEEE, 2018: 171−176.
19	ZHANG Z, GEIGER J, POHJALAINEN J, et al Deep learning for environmentally robust speech recognition: an overview of recent developments[J]. ACM Transactions on Intelligent Systems and Technology, 2018, 9 (5): 49
20	YU S, JIA S, XU C Convolutional neural networks for hyperspectral image classification[J]. Neurocomputing, 2017, 219: 88- 98 doi: 10.1016/j.neucom.2016.09.010
21	KHAN A, SOHAIL A, ZAHOORA U, et al A survey of the recent architectures of deep convolutional neural networks[J]. Artificial Intelligence Review, 2020, 53 (8): 5455- 5516 doi: 10.1007/s10462-020-09825-6
22	CHAIYAROJ A, SRI-IESARANUSORN P, BUEKBAN C, et al. Deep neural network approach for hand, wrist, grasping and functional movements classification using low-cost sEMG sensors [C]// 2019 IEEE International Conference on Bioinformatics and Biomedicine . San Diego: IEEE, 2019: 1443−1448.
23	TSINGANOS P, CORNELIS B, CORNELIS J, et al. Improved gesture recognition based on sEMG signals and TCN [C]// 2019 IEEE International Conference on Acoustics, Speech and Signal Processing . Brighton: IEEE, 2019: 1169−1173.
24	SHEN S, GU K, CHEN X, et al Gesture recognition through sEMG with wearable device based on deep learning[J]. Mobile Networks and Applications, 2020, 25: 2447- 2458 doi: 10.1007/s11036-020-01590-8
25	陈思佳, 罗志增基于长短时记忆和卷积神经网络的手势肌电识别研究[J]. 仪器仪表学报, 2021, 42 (2): 162- 170 CHEN Sijia, LUO Zhizeng Research on gesture EMG recognition based on long short-term memory and convolutional neural network[J]. Chinese Journal of Scientific Instrument, 2021, 42 (2): 162- 170
26	LI Y, YANG L, HE Z, et al Low-cost data glove based on deep-learning-enhanced flexible multiwalled carbon nanotube sensors for real-time gesture recognition[J]. Advanced Intelligent Systems, 2022, 4 (11): 2200128 doi: 10.1002/aisy.202200128
27	LI Y, ZHANG W, ZHANG Q, et al Transfer learning-based muscle activity decoding scheme by low-frequency sEMG for wearable low-cost application[J]. IEEE Access, 2021, 9: 22804- 22815 doi: 10.1109/ACCESS.2021.3056412
28	HE Y, FUKUDA O, BU N, et al. Surface EMG pattern recognition using long short-term memory combined with multilayer perceptron [C]// 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society . Honolulu: IEEE, 2018: 5636−5639.
29	TSINGANOS P, CORNELIS B, CORNELIS B, et al. Deep learning in EMG-based gesture recognition [C]// International Conference on Physiological Computing Systems . Seville: [s. n.], 2018: 107−114.
30	CASTELLINI C, VAN DER SMAGT P Surface EMG in advanced hand prosthetics[J]. Biol Cybern, 2009, 100 (1): 35- 47 doi: 10.1007/s00422-008-0278-1
31	FARINA D, JIANG N, REHBAUM H, et al The extraction of neural information from the surface EMG for the control of upper-limb prostheses: emerging avenues and challenges[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2014, 22 (4): 797- 809 doi: 10.1109/TNSRE.2014.2305111
32	SIMÃO M, MENDES N, GIBARU O, et al A review on electromyography decoding and pattern recognition for human-machine interaction[J]. IEEE Access, 2019, 7: 39564- 39582 doi: 10.1109/ACCESS.2019.2906584
33	XU H, XIONG A Advances and disturbances in sEMG-based intentions and movements recognition: a review[J]. IEEE Sensors Journal, 2021, 21 (12): 13019- 13028 doi: 10.1109/JSEN.2021.3068521
34	XIONG D, ZHANG D, ZHAO X, et al Deep learning for EMG-based human-machine interaction: a review[J]. IEEE/CAA Journal of Automatica Sinica, 2021, 8 (3): 512- 533 doi: 10.1109/JAS.2021.1003865
35	KETYKÓ I, KOVÁCS F, VARGA K Z. Domain adaptation for sEMG-based gesture recognition with recurrent neural networks [C]// 2019 International Joint Conference on Neural Networks . Budapest: IEEE, 2019: 1−7.
36	SOSIN I, KUDENKO D, SHPILMAN A. Continuous gesture recognition from sEMG sensor data with recurrent neural networks and adversarial domain adaptation [C]// 2018 15th International Conference on Control, Automation, Robotics and Vision . Singapore: IEEE, 2018: 1436−1441.
37	ATZORI M, GIJSBERTS A, CASTELLINI C, et al Electromyography data for non-invasive naturally-controlled robotic hand prostheses[J]. Scientific Data, 2014, 1: 140053 doi: 10.1038/sdata.2014.53
38	GENG W, DU Y, JIN W, et al Gesture recognition by instantaneous surface EMG images[J]. Scientific Reports, 2016, 6: 36571 doi: 10.1038/srep36571
39	HUDGINS B, PARKER P, SCOTT R N A new strategy for multifunction myoelectric control[J]. IEEE Transactions on Biomedical Engineering, 1993, 40 (1): 82- 94 doi: 10.1109/10.204774
40	HU J, SHEN L, SUN G. Squeeze-and-excitation networks [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City: IEEE, 2018: 7132−7141.
41	ATREY P K, HOSSAIN M A, EI SADDIK A, et al Multimodal fusion for multimedia analysis: a survey[J]. Multimedia Systems, 2010, 16: 345- 379 doi: 10.1007/s00530-010-0182-0
42	ZHU Y, ZHUANG F, WANG J, et al Deep subdomain adaptation network for image classification[J]. IEEE Transactions on Neural Networks and Learning Systems, 2021, 32 (4): 1713- 1722 doi: 10.1109/TNNLS.2020.2988928
43	DU Y, WONG Y, JIN W, et. al. Semi-supervised learning for surface EMG-based gesture recognition [C]// Proceedings of the 26th International Joint Conference on Artificial Intelligence . Melbourne: AAAI Press, 2017: 1624–1630.
44	XU Z, YU J, XIANG W, et al A novel SE-CNN attention architecture for sEMG-based hand gesture recognition[J]. Computer Modeling in Engineering and Sciences, 2023, 134 (1): 157- 177 doi: 10.32604/cmes.2022.020035
45	JOSEPHS D, DRAKE C, HEROY A, et al sEMG gesture recognition with a simple model of attention[J]. Machine Learning for Health, 2020, 136: 126- 138
46	PENG F, CHEN C, LV D, et al Gesture recognition by ensemble extreme learning machine based on surface electromyography signals[J]. Frontiers in Human Neuroscience, 2022, 16: 1- 14

[1]	李卓峰,孙铭会. 基于高采样率惯性测量单元的手部状态与手势识别[J]. 浙江大学学报(工学版), 2023, 57(3): 503-511.
[2]	辛文斌,郝惠敏,卜明龙,兰媛,黄家海,熊晓燕. 基于ShuffleNetv2-YOLOv3模型的静态手势实时识别方法[J]. 浙江大学学报(工学版), 2021, 55(10): 1815-1824.
[3]	跃龙,陈岭,陈根才. 基于层次化BoF模型和Spectral-HIK过滤的手势识别算法闯[J]. J4, 2013, 47(9): 1531-1536.

Viewed

Full text

Abstract

Cited

Shared

Discussed