基于知识共享的遮挡人体姿态估计网络

doi:10.3785/j.issn.1008-973X.2024.10.003

浙江大学学报(工学版)

2024, Vol. 58

Issue (10): 2001-2010 DOI: 10.3785/j.issn.1008-973X.2024.10.003

计算机与控制工程

基于知识共享的遮挡人体姿态估计网络

江佳鸿(

),夏楠*(

),李长吾,于鑫淼

大连工业大学信息科学与工程学院，辽宁大连 116034

Occluded human pose estimation network based on knowledge sharing

Jiahong JIANG(

),Nan XIA*(

),Changwu LI,Xinmiao YU

School of Information Science and Engineering, Dalian Polytechnic University, Dalian 116034, China

全文: PDF(1801 KB) HTML

摘要：

现有人体姿态估计方法处理遮挡情况时性能较差，为此提出新的估计网络，包含遮挡区域强化卷积网络(OCNN)和遮挡特征补偿图卷积网络(OGCN). 设计高低阶特征匹配注意力以强化遮挡区域特征，由OCNN提取高适配权重，通过少量遮挡数据的方式实现遮挡部位的强化检测. 由OGCN消除障碍物特征，通过强化关键点共有及专有属性的方式补偿节点特征；进行邻接矩阵重要性加权以改善遮挡部位特征质量，提升检测精度. 所提网络在数据集COCO2017、COCO-Wholebody、CrowdPose上的检测精度分别为78.5%、67.1%、77.8%，优于对比算法. 在自建遮挡数据集上所提网络节约了75%的训练数据使用.

关键词： 人体姿态估计; 遮挡处理; 高低阶特征匹配; 节点特征补偿; 邻接矩阵加权

Abstract:

A new estimation network was proposed for improving the insufficient occlusion handling ability of existing human pose estimation methods. An occluded parts enhanced convolutional network (OCNN) and an occluded features compensation graph convolutional network (OGCN) were included in the proposed network. A high-low order feature matching attention was designed to strengthen the occlusion area features, and high-adaptation weights were extracted by OCNN, achieving enhanced detection of the occluded parts with a small amount of occlusion data. OGCN strengthened the shared and private attribute compensation node features by eliminating the obstacle features. The adjacency matrix was importance-weighted to enhance the quality of the occlusion area features and to improve the detection accuracy. The proposed network achieved detection accuracy of 78.5%, 67.1%, and 77.8% in the datasets COCO2017, COCO-Wholebody, and CrowdPose, respectively, outperforming the comparative algorithms. The proposed network saved 75% of the training data usage in the self-built occlusion dataset.

Key words: human pose estimation occlusion handling high-low order feature matching node feature compensation adjacency matrix weighting

收稿日期: 2024-03-26 出版日期: 2024-09-27

CLC:

TP 391.4

基金资助: 教育部产学合作协同育人资助项目（220603231024713）.

通讯作者: 夏楠 E-mail: jjh19990901@163.com;xianan@dlpu.edu.cn

作者简介: 江佳鸿（1999—），男，硕士生，从事人体姿态估计研究. orcid.org/0009-0006-2447-1968. E-mail：jjh19990901@163.com

	服务
	把本文推荐给朋友
	加入引用管理器
	E-mail Alert
	作者相关文章
	江佳鸿
	夏楠
	李长吾
	于鑫淼

引用本文:

江佳鸿,夏楠,李长吾,于鑫淼. 基于知识共享的遮挡人体姿态估计网络[J]. 浙江大学学报(工学版), 2024, 58(10): 2001-2010.

Jiahong JIANG,Nan XIA,Changwu LI,Xinmiao YU. Occluded human pose estimation network based on knowledge sharing. Journal of ZheJiang University (Engineering Science), 2024, 58(10): 2001-2010.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2024.10.003 或 https://www.zjujournals.com/eng/CN/Y2024/V58/I10/2001

图 1 基于知识共享的遮挡人体姿态估计网络流程图

图 2 高低阶特征匹配注意力流程图

图 3 关键点特征补偿和邻接矩阵重要性加权流程图

表 1 不同算法在COCO2017数据集上的性能对比

表 2 不同算法在COCO-Wholebody数据集上的性能对比

表 3 不同算法在CrowdPose数据集上的性能对比

表 4 不同算法的检测性能对比

图 4 不同算法的关键点热力图对比

图 5 基于知识共享的遮挡人体姿态估计网络的姿态估计效果图

图 6 不同算法的姿态估计效果图对比

1	孙雪菲, 张瑞峰, 关欣, 等强化先验骨架结构的轻量型高效人体姿态估计[J]. 浙江大学学报: 工学版, 2024, 58 (1): 50- 60 SUN Xuefei, ZHANG Ruifeng, GUAN Xin, et al Lightweight and efficient human pose estimation with enhanced priori skeleton structure[J]. Journal of Zhejiang University: Engineering Science, 2024, 58 (1): 50- 60
2	YU X W, CHEN G S. HRPoseFormer: high-resolution Transformer for human pose estimation via multi-scale token aggregation [C]// IEEE 16th International Conference on Solid-State and Integrated Circuit Technology . Nanjing: IEEE, 2022: 1–3.
3	ZHOU L, CHEN Y, WANG J Progressive direction-aware pose grammar for human pose estimation[J]. IEEE Transactions on Biometrics, Behavior, and Identity Science, 2023, 5 (4): 593- 605 doi: 10.1109/TBIOM.2023.3315509
4	ZHANG Z, LIU M, SHEN J, et al Lightweight whole body human pose estimation with two-stage refinement training strategy[J]. IEEE Transactions on Human-Machine Systems, 2024, 54 (1): 121- 130 doi: 10.1109/THMS.2024.3349652
5	LIN J, ZHENG Z, ZHONG Z, et al. Joint representation learning and keypoint detection for cross-view geo-localization [J]. IEEE Transactions on Image Processing , 2022, 31: 3780–3792.
6	MENG Q, QIN C, BAI W, et al MulViMotion: shape-aware 3D myocardial motion tracking from multi-view cardiac MRI[J]. IEEE Transactions on Medical Imaging, 2022, 41 (8): 1961- 1974 doi: 10.1109/TMI.2022.3154599
7	CHAKRAVARTHI B, PATIL A K, RYU J Y, et al Scenario-based sensed human motion editing and validation through the motion-sphere[J]. IEEE Access, 2022, 10: 28295- 28307 doi: 10.1109/ACCESS.2022.3157939
8	MARQUES B, SILVA S, ALVES J, et al A conceptual model and taxonomy for collaborative augmented reality[J]. IEEE Transactions on Visualization and Computer Graphics, 2022, 28 (12): 5113- 5133 doi: 10.1109/TVCG.2021.3101545
9	LI N, CHEN X, FENG Y, et al Human–computer interaction cognitive behavior modeling of command and control systems[J]. IEEE Internet of Things Journal, 2022, 9 (14): 12723- 12736 doi: 10.1109/JIOT.2021.3138247
10	SUN K, XIAO B, LIU D, et al. Deep high-resolution representation learning for human pose estimation [C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition . Long Beach: IEEE, 2019: 5693–5703.
11	WANG J, SUN K, CHENG T, et al Deep high-resolution representation learning for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43 (10): 3349- 3364 doi: 10.1109/TPAMI.2020.2983686
12	KE L, CHANG M C, QI H, et al DetPoseNet: improving multi-person pose estimation via coarse-pose filtering[J]. IEEE Transactions on Image Processing, 2022, 31: 2782- 2795 doi: 10.1109/TIP.2022.3161081
13	CAO Z, HIDALGO G, SIMON T, et al OpenPose: realtime multi-person 2D pose estimation using part affinity fields[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43 (1): 172- 186 doi: 10.1109/TPAMI.2019.2929257
14	LI Q, ZHANG Z, ZHANG F, et al HRNeXt: high-resolution context network for crowd pose estimation[J]. IEEE Transactions on Multimedia, 2023, 25 (46): 1521- 1528
15	KIM G, KIM H, KONG K, et al Human body aware feature extractor using attachable feature corrector for human pose estimation[J]. IEEE Transactions on Multimedia, 2023, 25: 5789- 5799 doi: 10.1109/TMM.2022.3199098
16	LIN T Y, MAIRE M, BELONGIE S, et al. Microsoft COCO: common objects in context [C]// European Conference on Computer Vision . [S. l.]: Springer, 2014: 740–755.
17	BANZI J, BULUGU I, YE Z Learning a deep predictive coding network for a semi-supervised 3D-hand pose estimation[J]. IEEE/CAA Journal of Automatica Sinica, 2020, 7 (5): 1371- 1379 doi: 10.1109/JAS.2020.1003090
18	KIM S, KANG S, CHOI H, et al Keypoint aware robust representation for transformer-based re-identification of occluded person[J]. IEEE Signal Processing Letters, 2023, 30: 65- 69 doi: 10.1109/LSP.2023.3240596
19	WANG Y J, LUO Y M, BAI G H, et al UformPose: a U-shaped hierarchical multi-scale keypoint-aware framework for human pose estimation[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2023, 33 (4): 1697- 1709 doi: 10.1109/TCSVT.2022.3213206
20	PENG S, ZHOU X, LIU Y, et al PVNet: pixel-wise voting network for 6DoF object pose estimation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44 (1): 3212- 3223
21	ARTACHO B, SAVAKIS A UniPose+: a unified framework for 2D and 3D human pose estimation in images and videos[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44 (12): 9641- 9653 doi: 10.1109/TPAMI.2021.3124736
22	GAI D, FENG R Y, MIN W, et al Spatiotemporal learning transformer for video-based human pose estimation[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2023, 33 (9): 4564- 4576 doi: 10.1109/TCSVT.2023.3269666
23	YIN Y, LIU M, ZHU Q, et al Multibranch attention graph convolutional networks for 3-D human pose estimation[J]. IEEE Transactions on Instrumentation and Measurement, 2023, 72: 2520412.
24	FAN J, ZHENG P, LI S, et al An integrated hand-object dense pose estimation approach with explicit occlusion awareness for human-robot collaborative disassembly[J]. IEEE Transactions on Automation Science and Engineering, 2024, 21 (1): 147- 156 doi: 10.1109/TASE.2022.3215584
25	PASA L, NAVARIN N, SPERDUTI A Multiresolution reservoir graph neural network[J]. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33 (6): 2642- 2653 doi: 10.1109/TNNLS.2021.3090503
26	MORSHED M G, SULTANA T, LEE Y K LeL-GNN: learnable edge sampling and line based graph neural network for link prediction[J]. IEEE Access, 2023, 11: 56083- 56097 doi: 10.1109/ACCESS.2023.3283029
27	ISUFI E, GAMA F, RIBEIRO A EdgeNets: edge varying graph neural networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44 (11): 7457- 7473 doi: 10.1109/TPAMI.2021.3111054
28	XU L, JIN S, LIU W, et al ZoomNAS: searching for whole-body human pose estimation in the wild[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45 (8): 5296- 5313
29	LEE K, KIM W, LEE S From human pose similarity metric to 3D human pose estimator: temporal propagating LSTM networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45 (2): 1781- 1797 doi: 10.1109/TPAMI.2022.3164344

[1]	孙雪菲,张瑞峰,关欣,李锵. 强化先验骨架结构的轻量型高效人体姿态估计[J]. 浙江大学学报(工学版), 2024, 58(1): 50-60.

Viewed

Full text

Abstract

Cited

Shared

Discussed