Please wait a minute...
浙江大学学报(工学版)  2025, Vol. 59 Issue (3): 504-511    DOI: 10.3785/j.issn.1008-973X.2025.03.008
计算机技术     
基于小样本人体运动行为识别的孪生网络算法
姚明辉1,2(),王悦燕2,吴启亮1,牛燕1,王聪1
1. 天津工业大学 航空航天学院,天津 300384
2. 天津工业大学 人工智能学院,天津 300384
Siamese networks algorithm based on small human motion behavior recognition
Minghui YAO1,2(),Yueyan WANG2,Qiliang WU1,Yan NIU1,Cong WANG1
1. School of Aeronautics and Astronautics, Tiangong University, Tianjin 300384, China
2. School of Artificial Intelligence, Tiangong University, Tianjin 300384, China
 全文: PDF(1040 KB)   HTML
摘要:

为了使用更少传感器实现更高精度的人体运动行为识别,提出基于DSC-BiGRU-Att孪生网络的方法,旨在使用加速度传感器通过深度学习网络模型,实现对人体运动行为的准确识别. 基于人体运动行为多分类任务目标,以孪生网络为基本框架,将预处理后的加速度传感器数据样本对输入孪生网络. 孪生网络由2个结构相同、参数共享的深度可分离网络(DSC)、双向门控循环单元(BiGRU)和注意力机制(Att)组成. 深度可分离卷积网络提取运动行为信号中的空间特征,双向门控单元提取信号中的时间特征,引入注意力机制对提取的特征进行强化,最终实现人体运动行为的识别. 利用真实的数据集对所提出的DSC-BiGRU-Att孪生网络进行验证. 实验结果表明,所提出的算法在样本量较少的情况下取得了较好的行为识别结果,改模型在自采集加速度数据集上的准确率为98.89%.

关键词: 深度学习人体运动行为识别注意力机制深度可分离卷积孪生网络    
Abstract:

A human motion behavior recognition method was proposed based on the DSC-BiGRU-Att Siamese networks, to address the challenge of achieving higher accuracy in human motion behavior recognition using fewer sensors. The method aimed to achieve accurate recognition of human motion behavior through deep learning network models using acceleration sensors. The pairs of pre-processed acceleration sensor data samples were inputted, based on the multi-classification task objective of human motion behavior, with Siamese networks as the basic framework. The Siamese networks consisted of two deeply separable convolutional (DSC) networks with identical structures and shared parameters, a bidirectional gated recurrent unit (BiGRU), and an attention mechanism (Att). The DSC network extracted spatial features from the motion behavior signal, while the BiGRU extracted temporal features. The attention mechanism was introduced to enhance the extracted features and ultimately facilitated human motion behavior recognition. Real-world datasets were utilized to validate the proposed DSC-BiGRU-Att Siamese networks. Experimental results demonstrated that the proposed algorithm achieved favorable behavior recognition results despite a small sample size, with an accuracy rate of 98.89% attained on the self-collected acceleration dataset.

Key words: deep learning    human motion behavior recognition    attention mechanism    deep separable convolution    Siamese networks
收稿日期: 2023-12-19 出版日期: 2025-03-10
CLC:  TP 391  
基金资助: 国家自然科学基金重点资助项目 (12232014).
作者简介: 姚明辉(1971—),女,教授,博士,从事振动能量俘获与转换研究. orcid.org/0000-0002-6313-268X. E-mail:merry_mingming@163.com
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章  
姚明辉
王悦燕
吴启亮
牛燕
王聪

引用本文:

姚明辉,王悦燕,吴启亮,牛燕,王聪. 基于小样本人体运动行为识别的孪生网络算法[J]. 浙江大学学报(工学版), 2025, 59(3): 504-511.

Minghui YAO,Yueyan WANG,Qiliang WU,Yan NIU,Cong WANG. Siamese networks algorithm based on small human motion behavior recognition. Journal of ZheJiang University (Engineering Science), 2025, 59(3): 504-511.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2025.03.008        https://www.zjujournals.com/eng/CN/Y2025/V59/I3/504

图 1  孪生网络基本结构
图 2  行为识别骨干网络结构
层名称核数量核大小输出维度
卷积1644$ \times $1125$ \times $64
池化10062$ \times $64
BN0062$ \times $64
卷积2322$ \times $161$ \times $32
池化20030$ \times $32
表 1  DSC模块主要结构参数
层名称M
CNNDSC
卷积1832268
池化100
BN1256256
卷积241282208
池化200
表 2  DSC模块结构下不同卷积方式参数量对比
图 3  GRU神经元结构
图 4  BiGRU模块结构
图 5  注意力机制模块结构
图 6  基于DSC-BiGRU-Att多损失孪生网络的人体行为识别模型整体结构
编号动作类别动作描述
1行走在平坦的路面上以不同的速度行走和转弯
2慢跑以不同的速度跑步
3跳跃以不同的高度原地跳跃
4上楼以不同的速度在不同的地方爬楼梯
5下楼以不同的速度下楼梯
6保持下肢静态站在固体表面上
(上肢可以有其他动作行为)
表 3  6种动作具体描述
图 7  数据采集过程
图 8  训练样本数量对行为识别结果的影响
PlanA/%P/%R/%F1
(a)88.2588.4688.360.8849
(b)89.0289.2189.320.8906
(c)90.8890.2590.560.9084
(d)90.8690.5690.490.9065
(e)80.5680.5980.740.8062
(f)81.3681.3281.350.8136
(g)85.6985.4885.390.8564
(h)91.8591.8491.680.9182
表 4  不同主干网络的实验结果
方案A/%P/%R/%F1
1)92.8192.7592.310.9250
2)94.8894.8294.800.9485
3)97.6497.7297.660.9766
4)98.8998.8098.890.9889
表 5  消融实验结果对比
1 AGGARWAL J K, RYOO M S Human activity analysis: a review[J]. ACM Computing Surveys, 2011, 43 (3): 16- 28
2 BEDDIAR D R , NINI B , SABOKROU M , et al. Vision-based human activity recognition: a survey[J]. Multimedia Tools and Applications , 2020, 79(41): 30509-30555.
3 LI Y, YANG G, SU Z, et al Human activity recognition based on multienvironment sensor data[J]. Information Fusion, 2023, 91: 47- 63
doi: 10.1016/j.inffus.2022.10.015
4 WANG W, LIU A X, SHAHZAD M, et al Device-free human activity recognition using commercial WiFi devices[J]. IEEE Journal on Selected Areas in Communications, 2017, 35 (5): 1118- 1131
doi: 10.1109/JSAC.2017.2679658
5 HALIM A, ABDELLATIF A, AWAD M I, et al Prediction of human gait activities using wearable sensors[J]. Proceedings of the Institution of Mechanical Engineers, Part H: Journal of Engineering in Medicine, 2021, 235 (6): 676- 687
6 ENGÜL G, OZCELIK E, MISRA S, et al Fusion of smartphone sensor data for classification of daily user activities[J]. Multimedia Tools and Applications, 2021, 80 (24): 33527- 33546
doi: 10.1007/s11042-021-11105-6
7 MASWADI K, GHANI N A, HAMID S, et al Human activity classification using Decision Tree and Naïve Bayes classifiers[J]. Multimedia Tools and Applications, 2021, 80 (14): 21709- 21726
doi: 10.1007/s11042-020-10447-x
8 WANG Z, CHEN Y Recognizing human concurrent activities using wearable sensors: a statistical modeling approach based on parallel HMM[J]. Sensor Review, 2017, 37 (3): 330- 337
doi: 10.1108/SR-01-2017-0003
9 TANG T, ZHENG L, WENG S, et al. Human activity recognition with smart watch based on H-SVM [M]// Frontier Computing: Theory, Technologies and Applications . Singapore: Springer, 2018: 179−186.
10 HUANG J, LIN S, WANG N, et al TSE-CNN: a two-stage end-to-end CNN for human activity recognition[J]. IEEE Journal of Biomedical and Health Informatics, 2020, 24 (1): 292- 299
11 NOORI F M, RIEGLER M, UDDIN M Z, et al Human activity recognition from multiple sensors data using multi-fusion representations and CNNs[J]. ACM Transactions on Multimedia Computing, Communications, and Applications, 2020, 16 (2): 1- 19
12 KAYA Y, TOPUZ E K Human activity recognition from multiple sensors data using deep CNNs[J]. Multimedia Tools and Applications, 2024, 83 (4): 10815- 10838
13 KIM C, LEE W Human activity recognition by the image type encoding method of 3-axial sensor data[J]. Applied Sciences, 2023, 13 (8): 4961
doi: 10.3390/app13084961
14 LI H, SHRESTHA A, HEIDARI H, et al Bi-LSTM network for multimodal continuous human activity recognition and fall detection[J]. IEEE Sensors Journal, 2020, 20 (3): 1191- 1201
15 VARSHNEY N, BAKARIYA B, KUSHWAHA A K S, et al Human activity recognition by combining external features with accelerometer sensor data using deep learning network model[J]. Multimedia Tools and Applications, 2022, 81 (24): 34633- 34652
doi: 10.1007/s11042-021-11313-0
16 UZUNHISARCIKLI E, KAVUNCUOĞLU E, ÖZDEMIR A T Investigating classification performance of hybrid deep learning and machine learning architectures on activity recognition[J]. Computational Intelligence, 2022, 38 (4): 1402- 1449
doi: 10.1111/coin.12517
17 XING Y, ZHU J, LI Y, et al An improved spatial temporal graph convolutional network for robust skeleton-based action recognition[J]. Applied Intelligence, 2023, 53 (4): 4592- 4608
doi: 10.1007/s10489-022-03589-y
18 SHI L, XU H, JI W, et al Real-time human activity recognition system based on capsule and LoRa[J]. IEEE Sensors Journal, 2021, 21 (1): 667- 677
19 USMAN SARWAR M, REHMAN JAVED A, KULSOOM F, et al PARCIV: recognizing physical activities having complex interclass variations using semantic data of smartphone[J]. Software: Practice and Experience, 2021, 51 (3): 532- 549
doi: 10.1002/spe.2846
20 ARSHAD H, KHAN M A, SHARIF M, et al Multi-level features fusion and selection for human gait recognition: an optimized framework of Bayesian model and binomial distribution[J]. International Journal of Machine Learning and Cybernetics, 2019, 10 (12): 3601- 3618
doi: 10.1007/s13042-019-00947-0
21 PANDEY A, ZEESHAN M, KUMAR S Csi-based joint location and activity monitoring for COVID-19 quarantine environments[J]. IEEE Sensors Journal, 2023, 23 (2): 969- 976
22 易子文, 孙中华, 冯金超, 等 用于行为识别的通道可分离卷积神经网络[J]. 信号处理, 2020, 36 (9): 1497- 1502
YI Ziwen, SUN Zhonghua, FENG Jinchao, et al Channel separable convolutional neural network for behavior recognition[J]. Signal Processing, 2020, 36 (9): 1497- 1502
23 ZHANG X, WANG Y, CHEN Y, et al. Short-term wind speed prediction based on GRU [C]// IEEE Sustainable Power and Energy Conference . Beijing: IEEE, 2019: 882-887.
[1] 刘登峰,郭文静,陈世海. 基于内容引导注意力的车道线检测网络[J]. 浙江大学学报(工学版), 2025, 59(3): 451-459.
[2] 薛雅丽,贺怡铭,崔闪,欧阳权. 基于改进YOLOv5的SAR图像有向舰船目标检测算法[J]. 浙江大学学报(工学版), 2025, 59(2): 261-268.
[3] 陈智超,杨杰,李凡,冯志成. 基于深度学习的列车运行环境感知关键算法研究综述[J]. 浙江大学学报(工学版), 2025, 59(1): 1-17.
[4] 杨冰,徐楚阳,姚金良,向学勤. 基于单目RGB图像的三维手部姿态估计方法[J]. 浙江大学学报(工学版), 2025, 59(1): 18-26.
[5] 刘登峰,陈世海,郭文静,柴志雷. 基于轻量残差网络的高效半色调算法[J]. 浙江大学学报(工学版), 2025, 59(1): 62-69.
[6] 于家艺,吴秦. 基于上下文信息增强和深度引导的单目3D目标检测[J]. 浙江大学学报(工学版), 2025, 59(1): 89-99.
[7] 赵顗,安醇,李铭浩,马健霄,怀硕. 城市快速路互通交织区车辆的换道持续距离选择[J]. 浙江大学学报(工学版), 2025, 59(1): 205-212.
[8] 李凡,杨杰,冯志成,陈智超,付云骁. 基于图像识别的弓网接触点检测方法[J]. 浙江大学学报(工学版), 2024, 58(9): 1801-1810.
[9] 肖力,曹志刚,卢浩冉,黄志坚,蔡袁强. 基于深度学习和梯度优化的弹性超材料设计[J]. 浙江大学学报(工学版), 2024, 58(9): 1892-1901.
[10] 李灿林,王新玥,马利庄,邵志文,张文娇. 融合注意力机制和结构线提取的图像卡通化[J]. 浙江大学学报(工学版), 2024, 58(8): 1728-1737.
[11] 李忠良,陈麒,石琳,杨朝,邹先明. 时间感知组合的动态知识图谱补全[J]. 浙江大学学报(工学版), 2024, 58(8): 1738-1747.
[12] 吴书晗,王丹,陈远方,贾子钰,张越棋,许萌. 融合注意力的滤波器组双视图图卷积运动想象脑电分类[J]. 浙江大学学报(工学版), 2024, 58(7): 1326-1335.
[13] 李林睿,王东升,范红杰. 基于法条知识的事理型类案检索方法[J]. 浙江大学学报(工学版), 2024, 58(7): 1357-1365.
[14] 马现伟,范朝辉,聂为之,李东,朱逸群. 对失效传感器具备鲁棒性的故障诊断方法[J]. 浙江大学学报(工学版), 2024, 58(7): 1488-1497.
[15] 杨军,张琛. 基于边界点估计与稀疏卷积神经网络的三维点云语义分割[J]. 浙江大学学报(工学版), 2024, 58(6): 1121-1132.