Siamese networks algorithm based on small human motion behavior recognition

doi:10.3785/j.issn.1008-973X.2025.03.008

Journal of ZheJiang University (Engineering Science)

2025, Vol. 59

Issue (3): 504-511 DOI: 10.3785/j.issn.1008-973X.2025.03.008

Siamese networks algorithm based on small human motion behavior recognition

Minghui YAO1,2(

),Yueyan WANG2,Qiliang WU1,Yan NIU1,Cong WANG1

1. School of Aeronautics and Astronautics, Tiangong University, Tianjin 300384, China
2. School of Artificial Intelligence, Tiangong University, Tianjin 300384, China

Download:

HTML

PDF(1040KB) HTML
Export: BibTeX | EndNote (RIS)

Abstract

A human motion behavior recognition method was proposed based on the DSC-BiGRU-Att Siamese networks, to address the challenge of achieving higher accuracy in human motion behavior recognition using fewer sensors. The method aimed to achieve accurate recognition of human motion behavior through deep learning network models using acceleration sensors. The pairs of pre-processed acceleration sensor data samples were inputted, based on the multi-classification task objective of human motion behavior, with Siamese networks as the basic framework. The Siamese networks consisted of two deeply separable convolutional (DSC) networks with identical structures and shared parameters, a bidirectional gated recurrent unit (BiGRU), and an attention mechanism (Att). The DSC network extracted spatial features from the motion behavior signal, while the BiGRU extracted temporal features. The attention mechanism was introduced to enhance the extracted features and ultimately facilitated human motion behavior recognition. Real-world datasets were utilized to validate the proposed DSC-BiGRU-Att Siamese networks. Experimental results demonstrated that the proposed algorithm achieved favorable behavior recognition results despite a small sample size, with an accuracy rate of 98.89% attained on the self-collected acceleration dataset.

Key words： deep learning human motion behavior recognition attention mechanism deep separable convolution Siamese networks

Received: 19 December 2023 Published: 10 March 2025

CLC:

TP 391

Fund: 国家自然科学基金重点资助项目 (12232014).

	Service
	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	Minghui YAO
	Yueyan WANG
	Qiliang WU
	Yan NIU
	Cong WANG

Cite this article:

Minghui YAO,Yueyan WANG,Qiliang WU,Yan NIU,Cong WANG. Siamese networks algorithm based on small human motion behavior recognition. Journal of ZheJiang University (Engineering Science), 2025, 59(3): 504-511.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2025.03.008 OR https://www.zjujournals.com/eng/Y2025/V59/I3/504

基于小样本人体运动行为识别的孪生网络算法

为了使用更少传感器实现更高精度的人体运动行为识别，提出基于DSC-BiGRU-Att孪生网络的方法，旨在使用加速度传感器通过深度学习网络模型，实现对人体运动行为的准确识别. 基于人体运动行为多分类任务目标，以孪生网络为基本框架，将预处理后的加速度传感器数据样本对输入孪生网络. 孪生网络由2个结构相同、参数共享的深度可分离网络(DSC)、双向门控循环单元(BiGRU)和注意力机制(Att)组成. 深度可分离卷积网络提取运动行为信号中的空间特征，双向门控单元提取信号中的时间特征，引入注意力机制对提取的特征进行强化，最终实现人体运动行为的识别. 利用真实的数据集对所提出的DSC-BiGRU-Att孪生网络进行验证. 实验结果表明，所提出的算法在样本量较少的情况下取得了较好的行为识别结果，改模型在自采集加速度数据集上的准确率为98.89%.

关键词： 深度学习, 人体运动行为识别, 注意力机制, 深度可分离卷积, 孪生网络

Fig.1 Structure of Siamese networks

Fig.2 Activity recognition backbone network architecture

Tab.1 Main structural parameters of DSC module

Tab.2 Comparison of parametric quantities for different convolution methods under DSC module structure

Fig.3 Structure of GRU neurons

Fig.4 BiGRU module structure

Fig.5 Structure of attention mechanism module

Fig.6 Structure of human behavior recognition model based on DSC-BiGRU-Att multi-loss Siamese networks

Tab.3 Specific description of six actions

Fig.7 Data collection process

Fig.8 Effect of number of training samples on behavioral recognition results

Tab.4 Experimental results of different backbone networks

Tab.5 Comparison of ablation experiment results


[1]	AGGARWAL J K, RYOO M S Human activity analysis: a review[J]. ACM Computing Surveys, 2011, 43 (3): 16- 28

[2]	BEDDIAR D R , NINI B , SABOKROU M , et al. Vision-based human activity recognition: a survey[J]. Multimedia Tools and Applications , 2020, 79(41): 30509-30555.

[3]	LI Y, YANG G, SU Z, et al Human activity recognition based on multienvironment sensor data[J]. Information Fusion, 2023, 91: 47- 63 doi: 10.1016/j.inffus.2022.10.015

[4]	WANG W, LIU A X, SHAHZAD M, et al Device-free human activity recognition using commercial WiFi devices[J]. IEEE Journal on Selected Areas in Communications, 2017, 35 (5): 1118- 1131 doi: 10.1109/JSAC.2017.2679658

[5]	HALIM A, ABDELLATIF A, AWAD M I, et al Prediction of human gait activities using wearable sensors[J]. Proceedings of the Institution of Mechanical Engineers, Part H: Journal of Engineering in Medicine, 2021, 235 (6): 676- 687

[6]	ENGÜL G, OZCELIK E, MISRA S, et al Fusion of smartphone sensor data for classification of daily user activities[J]. Multimedia Tools and Applications, 2021, 80 (24): 33527- 33546 doi: 10.1007/s11042-021-11105-6

[7]	MASWADI K, GHANI N A, HAMID S, et al Human activity classification using Decision Tree and Naïve Bayes classifiers[J]. Multimedia Tools and Applications, 2021, 80 (14): 21709- 21726 doi: 10.1007/s11042-020-10447-x

[8]	WANG Z, CHEN Y Recognizing human concurrent activities using wearable sensors: a statistical modeling approach based on parallel HMM[J]. Sensor Review, 2017, 37 (3): 330- 337 doi: 10.1108/SR-01-2017-0003

[9]	TANG T, ZHENG L, WENG S, et al. Human activity recognition with smart watch based on H-SVM [M]// Frontier Computing: Theory, Technologies and Applications . Singapore: Springer, 2018: 179−186.

[10]	HUANG J, LIN S, WANG N, et al TSE-CNN: a two-stage end-to-end CNN for human activity recognition[J]. IEEE Journal of Biomedical and Health Informatics, 2020, 24 (1): 292- 299

[11]	NOORI F M, RIEGLER M, UDDIN M Z, et al Human activity recognition from multiple sensors data using multi-fusion representations and CNNs[J]. ACM Transactions on Multimedia Computing, Communications, and Applications, 2020, 16 (2): 1- 19

[12]	KAYA Y, TOPUZ E K Human activity recognition from multiple sensors data using deep CNNs[J]. Multimedia Tools and Applications, 2024, 83 (4): 10815- 10838

[13]	KIM C, LEE W Human activity recognition by the image type encoding method of 3-axial sensor data[J]. Applied Sciences, 2023, 13 (8): 4961 doi: 10.3390/app13084961

[14]	LI H, SHRESTHA A, HEIDARI H, et al Bi-LSTM network for multimodal continuous human activity recognition and fall detection[J]. IEEE Sensors Journal, 2020, 20 (3): 1191- 1201

[15]	VARSHNEY N, BAKARIYA B, KUSHWAHA A K S, et al Human activity recognition by combining external features with accelerometer sensor data using deep learning network model[J]. Multimedia Tools and Applications, 2022, 81 (24): 34633- 34652 doi: 10.1007/s11042-021-11313-0

[16]	UZUNHISARCIKLI E, KAVUNCUOĞLU E, ÖZDEMIR A T Investigating classification performance of hybrid deep learning and machine learning architectures on activity recognition[J]. Computational Intelligence, 2022, 38 (4): 1402- 1449 doi: 10.1111/coin.12517

[17]	XING Y, ZHU J, LI Y, et al An improved spatial temporal graph convolutional network for robust skeleton-based action recognition[J]. Applied Intelligence, 2023, 53 (4): 4592- 4608 doi: 10.1007/s10489-022-03589-y

[18]	SHI L, XU H, JI W, et al Real-time human activity recognition system based on capsule and LoRa[J]. IEEE Sensors Journal, 2021, 21 (1): 667- 677

[19]	USMAN SARWAR M, REHMAN JAVED A, KULSOOM F, et al PARCIV: recognizing physical activities having complex interclass variations using semantic data of smartphone[J]. Software: Practice and Experience, 2021, 51 (3): 532- 549 doi: 10.1002/spe.2846

[20]	ARSHAD H, KHAN M A, SHARIF M, et al Multi-level features fusion and selection for human gait recognition: an optimized framework of Bayesian model and binomial distribution[J]. International Journal of Machine Learning and Cybernetics, 2019, 10 (12): 3601- 3618 doi: 10.1007/s13042-019-00947-0

[21]	PANDEY A, ZEESHAN M, KUMAR S Csi-based joint location and activity monitoring for COVID-19 quarantine environments[J]. IEEE Sensors Journal, 2023, 23 (2): 969- 976

[22]	易子文, 孙中华, 冯金超, 等用于行为识别的通道可分离卷积神经网络[J]. 信号处理, 2020, 36 (9): 1497- 1502 YI Ziwen, SUN Zhonghua, FENG Jinchao, et al Channel separable convolutional neural network for behavior recognition[J]. Signal Processing, 2020, 36 (9): 1497- 1502

[23]	ZHANG X, WANG Y, CHEN Y, et al. Short-term wind speed prediction based on GRU [C]// IEEE Sustainable Power and Energy Conference . Beijing: IEEE, 2019: 882-887.

[1]	Dengfeng LIU,Wenjing GUO,Shihai CHEN. Content-guided attention-based lane detection network[J]. Journal of ZheJiang University (Engineering Science), 2025, 59(3): 451-459.

[2]	Yali XUE,Yiming HE,Shan CUI,Quan OUYANG. Oriented ship detection algorithm in SAR image based on improved YOLOv5[J]. Journal of ZheJiang University (Engineering Science), 2025, 59(2): 261-268.

[3]	Zhichao CHEN,Jie YANG,Fan LI,Zhicheng FENG. Review on deep learning-based key algorithm for train running environment perception[J]. Journal of ZheJiang University (Engineering Science), 2025, 59(1): 1-17.

[4]	Dengfeng LIU,Shihai CHEN,Wenjing GUO,Zhilei CHAI. Efficient halftone algorithm based on lightweight residual networks[J]. Journal of ZheJiang University (Engineering Science), 2025, 59(1): 62-69.

[5]	Yi ZHAO,Chun AN,Minghao LI,Jianxiao MA,Shuo HUAI. Selection of lane-changing distance for vehicles in urban expressway interchange weaving section[J]. Journal of ZheJiang University (Engineering Science), 2025, 59(1): 205-212.

[6]	Fan LI,Jie YANG,Zhicheng FENG,Zhichao CHEN,Yunxiao FU. Pantograph-catenary contact point detection method based on image recognition[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(9): 1801-1810.

[7]	Li XIAO,Zhigang CAO,Haoran LU,Zhijian HUANG,Yuanqiang CAI. Elastic metamaterial design based on deep learning and gradient optimization[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(9): 1892-1901.

[8]	Canlin LI,Xinyue WANG,Lizhuang MA,Zhiwen SHAO,Wenjiao ZHANG. Image cartoonization incorporating attention mechanism and structural line extraction[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(8): 1728-1737.

[9]	Zhongliang LI,Qi CHEN,Lin SHI,Chao YANG,Xianming ZOU. Dynamic knowledge graph completion of temporal aware combination[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(8): 1738-1747.

[10]	Shuhan WU,Dan WANG,Yuanfang CHEN,Ziyu JIA,Yueqi ZHANG,Meng XU. Attention-fused filter bank dual-view graph convolution motor imagery EEG classification[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(7): 1326-1335.

[11]	Linrui LI,Dongsheng WANG,Hongjie FAN. Fact-based similar case retrieval methods based on statutory knowledge[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(7): 1357-1365.

[12]	Xianwei MA,Chaohui FAN,Weizhi NIE,Dong LI,Yiqun ZHU. Robust fault diagnosis method for failure sensors[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(7): 1488-1497.

[13]	Jun YANG,Chen ZHANG. Semantic segmentation of 3D point cloud based on boundary point estimation and sparse convolution neural network[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(6): 1121-1132.

[14]	Yuntang LI,Hengjie LI,Kun ZHANG,Binrui WANG,Shanyue GUAN,Yuan CHEN. Recognition of complex power lines based on novel encoder-decoder network[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(6): 1133-1141.

[15]	Juan SONG,Longxi HE,Huiping LONG. Deep learning-based algorithm for multi defect detection in tunnel lining[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(6): 1161-1173.

Viewed

Full text

Abstract

Cited

Shared

Discussed