Convolutional neural network combined with subdomain adaptation for low sampling rate EMG-based gesture recognition

doi:10.3785/j.issn.1008-973X.2024.10.004

Journal of ZheJiang University (Engineering Science)

2024, Vol. 58

Issue (10): 2011-2019 DOI: 10.3785/j.issn.1008-973X.2024.10.004

Convolutional neural network combined with subdomain adaptation for low sampling rate EMG-based gesture recognition

Diao ZHOU(

),Xin XIONG,Jianhua ZHOU*(

),Jing ZONG,Qi ZHANG

Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, China

Download:

HTML

PDF(1146KB) HTML
Export: BibTeX | EndNote (RIS)

Abstract

A new recognition method was proposed to improve the performance of low sampling rate electromyography (EMG)-based gesture recognition. The information of the pre-processed low sampling rate EMG signal was extended by an information extension layer, and the representation of key features was enhanced. In the feature extraction network, domain invariant features in the source and target domains were extracted by the subdomain adaptation network, then the domain invariant features were classified. The proposed method was evaluated using the DB1 and DB5 sub-databases of the NinaPro database. Experimental results showed that the proposed method recognized 53 and 52 gestures with the highest accuracy of 90.89% (DB1), 89.90% (DB5) and 82.01% (DB1), 77.07% (DB5), respectively. The effects of factors on low sampling rate EMG-based gesture recognition are reduced by the proposed method, factors that include electrode shift, muscle fatigue, changes in skin impedance, and the relative movement of the muscle relative to the electrodes.

Key words： low sampling rate surface electromyography gesture recognition subdomain adaptation information expansion squeeze-and-excitation attention mechanism

Received: 12 September 2023 Published: 27 September 2024

CLC:

TP 391

Fund: 国家自然科学基金资助项目（82060329）.

Corresponding Authors: Jianhua ZHOU E-mail: 1962589274@qq.com;742028837@qq.com

	Service
	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	Diao ZHOU
	Xin XIONG
	Jianhua ZHOU
	Jing ZONG
	Qi ZHANG

Cite this article:

Diao ZHOU,Xin XIONG,Jianhua ZHOU,Jing ZONG,Qi ZHANG. Convolutional neural network combined with subdomain adaptation for low sampling rate EMG-based gesture recognition. Journal of ZheJiang University (Engineering Science), 2024, 58(10): 2011-2019.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2024.10.004 OR https://www.zjujournals.com/eng/Y2024/V58/I10/2011

卷积神经网络结合子域适应的低采样率肌电手势识别

为了提升模型识别低采样率肌电手势的性能，提出新的识别方法. 通过信息扩展层对预处理后的低采样率肌电信号信息进行扩展，增强关键特征的表示. 在特征提取网络中，利用子域适应网络提取源域与目标域中的域不变特征后进行域不变特征分类. 使用NinaPro数据库中的DB1和DB5子数据库对所提方法进行评估. 实验结果表明，所提方法识别53种和52种手势的最高准确率分别为90.89%(DB1)、89.90%(DB5)和82.01%(DB1)、77.07%(DB5)，能够降低电极移位、肌肉疲劳、皮肤阻抗的变化和肌肉相对电极的相对运动等因素对低采样率肌电手势识别的影响.

关键词： 低采样率表面肌电, 手势识别, 子域适应, 信息扩展, 挤压与激励注意力机制

Tab.1 DB1 and DB5 sub-database details in NinaPro

Fig.1 Structure of low sampling rate surface electromyography-based gesture recognition model

Tab.2 Parameters of convolutional neural network

Fig.2 Structure of information expansion layer in low sampling rate surface electromyography-based gesture recognition model

Tab.3 Convolutional layer parameters in information expansion layer

Fig.3 Structure of squeeze-and-excitation attention mechanism in information expansion layer

Fig.4 Structure of subdomain adaptation network

Fig.5 Diagram of results of hyperparametric experiment

Tab.4 Recognition accuracy of 53 gestures with different weight parameters

Fig.6 Confusion matrix for classification results of 53 gestures

Tab.5 Recognition accuracy of 52 gestures with different weight parameters

Fig.7 Confusion matrix for classification results of 52 gestures

Fig.8 Data distribution of source domain and target domain and feature distribution of two models

Tab.6 Recognition accuracy of different types of gesture actions

Tab.7 Performance comparison with different gesture recognition methods

Tab.8 Ablation experiments for model structure performance


[1]	LI K, ZHANG J, WANG L, et al A review of the key technologies for sEMG-based human-robot interaction systems[J]. Biomedical Signal Processing and Control, 2020, 62: 102074 doi: 10.1016/j.bspc.2020.102074

[2]	JIANG S, KANG P, SONG X, et al Emerging wearable interfaces and algorithms for hand gesture recognition: a survey[J]. IEEE Reviews in Biomedical Engineering, 2021, 15: 85- 102

[3]	NGUYEN-TRONG K, VU H N, TRUNG N N, et al Gesture recognition using wearable sensors with bi-long short-term memory convolutional neural networks[J]. IEEE Sensors Journal, 2021, 21 (13): 15065- 15079 doi: 10.1109/JSEN.2021.3074642

[4]	JIANG X, XU K, LIU X, et al Neuromuscular password-based user authentication[J]. IEEE Transactions on Industrial Informatics, 2021, 17 (4): 2641- 2652 doi: 10.1109/TII.2020.3001612

[5]	CHEN X, GONG L, ZHENG L, et al. Soft exoskeleton glove for hand assistance based on human-machine interaction and machine learning [C]// 2020 IEEE International Conference on Human-Machine Systems . Rome: IEEE, 2020: 1−6.

[6]	Al-FAHAAM H, DAVIS S, NEFTI-MEZIANI S, et al Novel soft bending actuator-based power augmentation hand exoskeleton controlled by human intention[J]. Intelligent Service Robotics, 2018, 11 (3): 247- 268 doi: 10.1007/s11370-018-0250-4

[7]	PHINYOMARK A, KHUSHABA R N, SCHEME E Feature extraction and selection for myoelectric control based on wearable EMG sensors[J]. Sensors, 2018, 18 (5): 1615 doi: 10.3390/s18051615

[8]	CLANCT E A, MORIN E L, MERLETTI R Sampling, noise-reduction and amplitude estimation issues in surface electromyography[J]. Journal of Electromyography and Kinesiology, 2002, 12 (1): 1- 16 doi: 10.1016/S1050-6411(01)00033-5

[9]	IVES J C, WIGGLESWORTH J K Sampling rate effects on surface EMG timing and amplitude measures[J]. Clinical Biomechanics, 2003, 18 (6): 543- 552 doi: 10.1016/S0268-0033(03)00089-5

[10]	LI G, LI Y, YU L, et al Conditioning and sampling issues of EMG signals in motion recognition of multifunctional myoelectric prostheses[J]. Annals of Biomedical Engineering, 2011, 39 (6): 1779- 1787 doi: 10.1007/s10439-011-0265-x

[11]	PHINYOMARK A, SCHEME E. A feature extraction issue for myoelectric control based on wearable EMG sensors [C]// 2018 IEEE Sensors Applications Symposium . Seoul: IEEE, 2018: 1−6.

[12]	CHEN H, ZHANG Y, LI G, et al Surface electromyography feature extraction via convolutional neural network[J]. International Journal of Machine Learning and Cybernetics, 2020, 11: 185- 196 doi: 10.1007/s13042-019-00966-x

[13]	ATZORI M, GIJSBERTS A, HEYNEN S, et al. Building the Ninapro database: a resource for the biorobotics community [C]// 2012 4th IEEE RAS and EMBS International Conference on Biomedical Robotics and Biomechatronics . Rome: IEEE, 2012: 1258−1265.

[14]	CAO L, ZHANG W, KAN X, et al A novel adaptive mutation PSO optimized SVM algorithm for sEMG-based gesture recognition[J]. Scientific Programming, 2021, 2021 (1): 9988823

[15]	WAHID M F, TAFRESHI R, AL-SOWAIDI M, et al Subject-independent hand gesture recognition using normalization and machine learning algorithms[J]. Journal of Computational Science, 2018, 27: 69- 76 doi: 10.1016/j.jocs.2018.04.019

[16]	TEPE C, ERDIM M Classification of surface electromyography and gyroscopic signals of finger gestures acquired by Myo armband using machine learning methods[J]. Biomedical Signal Processing and Control, 2022, 75: 103588 doi: 10.1016/j.bspc.2022.103588

[17]	JARAMILLO-YANEZ A, UNAPANTA L, BENALCÁZAR M E. Short-term hand gesture recognition using electromyography in the transient state, support vector machines, and discrete wavelet transform [C]// 2019 IEEE Latin American Conference on Computational Intelligence . Guayaquil: IEEE, 2019: 1−6.

[18]	WAHID M F, TAFRESHI R, AL-SOWAIDI M, et al. An efficient approach to recognize hand gestures using machine-learning algorithms [C]// 2018 IEEE 4th Middle East Conference on Biomedical Engineering . Tunis: IEEE, 2018: 171−176.

[19]	ZHANG Z, GEIGER J, POHJALAINEN J, et al Deep learning for environmentally robust speech recognition: an overview of recent developments[J]. ACM Transactions on Intelligent Systems and Technology, 2018, 9 (5): 49

[20]	YU S, JIA S, XU C Convolutional neural networks for hyperspectral image classification[J]. Neurocomputing, 2017, 219: 88- 98 doi: 10.1016/j.neucom.2016.09.010

[21]	KHAN A, SOHAIL A, ZAHOORA U, et al A survey of the recent architectures of deep convolutional neural networks[J]. Artificial Intelligence Review, 2020, 53 (8): 5455- 5516 doi: 10.1007/s10462-020-09825-6

[22]	CHAIYAROJ A, SRI-IESARANUSORN P, BUEKBAN C, et al. Deep neural network approach for hand, wrist, grasping and functional movements classification using low-cost sEMG sensors [C]// 2019 IEEE International Conference on Bioinformatics and Biomedicine . San Diego: IEEE, 2019: 1443−1448.

[23]	TSINGANOS P, CORNELIS B, CORNELIS J, et al. Improved gesture recognition based on sEMG signals and TCN [C]// 2019 IEEE International Conference on Acoustics, Speech and Signal Processing . Brighton: IEEE, 2019: 1169−1173.

[24]	SHEN S, GU K, CHEN X, et al Gesture recognition through sEMG with wearable device based on deep learning[J]. Mobile Networks and Applications, 2020, 25: 2447- 2458 doi: 10.1007/s11036-020-01590-8

[25]	陈思佳, 罗志增基于长短时记忆和卷积神经网络的手势肌电识别研究[J]. 仪器仪表学报, 2021, 42 (2): 162- 170 CHEN Sijia, LUO Zhizeng Research on gesture EMG recognition based on long short-term memory and convolutional neural network[J]. Chinese Journal of Scientific Instrument, 2021, 42 (2): 162- 170

[26]	LI Y, YANG L, HE Z, et al Low-cost data glove based on deep-learning-enhanced flexible multiwalled carbon nanotube sensors for real-time gesture recognition[J]. Advanced Intelligent Systems, 2022, 4 (11): 2200128 doi: 10.1002/aisy.202200128

[27]	LI Y, ZHANG W, ZHANG Q, et al Transfer learning-based muscle activity decoding scheme by low-frequency sEMG for wearable low-cost application[J]. IEEE Access, 2021, 9: 22804- 22815 doi: 10.1109/ACCESS.2021.3056412

[28]	HE Y, FUKUDA O, BU N, et al. Surface EMG pattern recognition using long short-term memory combined with multilayer perceptron [C]// 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society . Honolulu: IEEE, 2018: 5636−5639.

[29]	TSINGANOS P, CORNELIS B, CORNELIS B, et al. Deep learning in EMG-based gesture recognition [C]// International Conference on Physiological Computing Systems . Seville: [s. n.], 2018: 107−114.

[30]	CASTELLINI C, VAN DER SMAGT P Surface EMG in advanced hand prosthetics[J]. Biol Cybern, 2009, 100 (1): 35- 47 doi: 10.1007/s00422-008-0278-1

[31]	FARINA D, JIANG N, REHBAUM H, et al The extraction of neural information from the surface EMG for the control of upper-limb prostheses: emerging avenues and challenges[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2014, 22 (4): 797- 809 doi: 10.1109/TNSRE.2014.2305111

[32]	SIMÃO M, MENDES N, GIBARU O, et al A review on electromyography decoding and pattern recognition for human-machine interaction[J]. IEEE Access, 2019, 7: 39564- 39582 doi: 10.1109/ACCESS.2019.2906584

[33]	XU H, XIONG A Advances and disturbances in sEMG-based intentions and movements recognition: a review[J]. IEEE Sensors Journal, 2021, 21 (12): 13019- 13028 doi: 10.1109/JSEN.2021.3068521

[34]	XIONG D, ZHANG D, ZHAO X, et al Deep learning for EMG-based human-machine interaction: a review[J]. IEEE/CAA Journal of Automatica Sinica, 2021, 8 (3): 512- 533 doi: 10.1109/JAS.2021.1003865

[35]	KETYKÓ I, KOVÁCS F, VARGA K Z. Domain adaptation for sEMG-based gesture recognition with recurrent neural networks [C]// 2019 International Joint Conference on Neural Networks . Budapest: IEEE, 2019: 1−7.

[36]	SOSIN I, KUDENKO D, SHPILMAN A. Continuous gesture recognition from sEMG sensor data with recurrent neural networks and adversarial domain adaptation [C]// 2018 15th International Conference on Control, Automation, Robotics and Vision . Singapore: IEEE, 2018: 1436−1441.

[37]	ATZORI M, GIJSBERTS A, CASTELLINI C, et al Electromyography data for non-invasive naturally-controlled robotic hand prostheses[J]. Scientific Data, 2014, 1: 140053 doi: 10.1038/sdata.2014.53

[38]	GENG W, DU Y, JIN W, et al Gesture recognition by instantaneous surface EMG images[J]. Scientific Reports, 2016, 6: 36571 doi: 10.1038/srep36571

[39]	HUDGINS B, PARKER P, SCOTT R N A new strategy for multifunction myoelectric control[J]. IEEE Transactions on Biomedical Engineering, 1993, 40 (1): 82- 94 doi: 10.1109/10.204774

[40]	HU J, SHEN L, SUN G. Squeeze-and-excitation networks [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City: IEEE, 2018: 7132−7141.

[41]	ATREY P K, HOSSAIN M A, EI SADDIK A, et al Multimodal fusion for multimedia analysis: a survey[J]. Multimedia Systems, 2010, 16: 345- 379 doi: 10.1007/s00530-010-0182-0

[42]	ZHU Y, ZHUANG F, WANG J, et al Deep subdomain adaptation network for image classification[J]. IEEE Transactions on Neural Networks and Learning Systems, 2021, 32 (4): 1713- 1722 doi: 10.1109/TNNLS.2020.2988928

[43]	DU Y, WONG Y, JIN W, et. al. Semi-supervised learning for surface EMG-based gesture recognition [C]// Proceedings of the 26th International Joint Conference on Artificial Intelligence . Melbourne: AAAI Press, 2017: 1624–1630.

[44]	XU Z, YU J, XIANG W, et al A novel SE-CNN attention architecture for sEMG-based hand gesture recognition[J]. Computer Modeling in Engineering and Sciences, 2023, 134 (1): 157- 177 doi: 10.32604/cmes.2022.020035

[45]	JOSEPHS D, DRAKE C, HEROY A, et al sEMG gesture recognition with a simple model of attention[J]. Machine Learning for Health, 2020, 136: 126- 138

[46]	PENG F, CHEN C, LV D, et al Gesture recognition by ensemble extreme learning machine based on surface electromyography signals[J]. Frontiers in Human Neuroscience, 2022, 16: 1- 14

[1]	Zhuo-feng LI,Ming-hui SUN. Hand gesture/state recognition based on inertial measurement unit at high sample rate[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(3): 503-511.

[2]	Wen-bin XIN,Hui-min HAO,Ming-long BU,Yuan LAN,Jia-hai HUANG,Xiao-yan XIONG. Static gesture real-time recognition method based on ShuffleNetv2-YOLOv3 model[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(10): 1815-1824.

Viewed

Full text

Abstract

Cited

Shared

Discussed