Please wait a minute...
浙江大学学报(工学版)
浙江大学学报(工学版)  2021, Vol. 55 Issue (11): 2022-2032    DOI: 10.3785/j.issn.1008-973X.2021.11.002
计算机技术     
现实场景下行人身份识别
闫禹铭1(),何剑锋2,李殊昭1,于慧敏1,*(),王锋2,任周丰2
1. 浙江大学 信息与电子工程学院，浙江 杭州 310007
2. 江南社会学院，江苏 苏州 215124
Identity recognition under real scenes
Yu-ming YAN1(),Jian-feng HE2,Shu-zhao LI1,Hui-min YU1,*(),Feng WANG2,Zhou-feng REN2
1. College of Information Science and Electronic Engineering, Zhejiang University, Hangzhou 310007, China
2. Jiangnan Social University, Suzhou 215124, China
 全文: PDF(966 KB)   HTML
摘要：

为了实现现实场景下的行人身份识别，结合人脸识别和行人重识别（Re-id）技术设计行人身份识别系统. 人脸识别技术能够在检测到行人的人脸时提高行人身份识别的精度，行人重识别通过改进的衣着信息消除（ICESD）模块来解决现实场景下的行人换装问题. 在存在大量未知行人的现实场景下，通过先识别是否为已知行人，然后识别具体行人类别的阈值过滤识别过程显著提升准度. 现实场景的应用测试和比较试验表明，相较于主流深度学习方法，在换装场景下，行人重识别技术能实现性能的显著提升，人脸识别也能显著提升整体识别准度. 系统的实时性分析表明，行人身份系统可以布置在现实场景下.
关键词： 人脸识别;  换装行人重识别;  现实场景;  行人身份识别;  阈值过滤    
Abstract:

An identity recognition system combing face recognition and person Re-identification (Re-id) was designed for identity recognition under real scenes. When the pedestrian ’s face was detected, the recognition accuracy of the system was significantly improved through the face recognition technology. Improved cloth-elimination shape-distillation (ICESD) module was used to solve the problem of cloth-changing of pedestrians. In a real scene with a large number of unknown pedestrians, the recognition accuracy is significantly improved through the threshold filtering recognition process of identifying the known pedestrian first and then the specific pedestrian category. The application tests and comparative tests in real scenes demonstrate that the proposed Re-id method achieves significant performance improvement compared with mainstream deep learning methods under cloth-changing scenarios, and face recognition technology can also significantly improve the overall recognition accuracy. In the meantime, the real-time analysis of the system shows that the system can be deployed under real scenes.
Key words: face recognition    cloth-changing person Re-identification    real scene    identity recognition    threshold filtering
收稿日期: 2021-03-22 出版日期: 2021-11-05
CLC:  TP 183  
基金资助: 之江实验室开放基金资助项目（2019KD0AB01）
通讯作者: 于慧敏     E-mail: 21931092@zju.edu.cn;yhm2005@zju.edu.cn
作者简介: 闫禹铭（1998—），男，硕士，从事计算机视觉研究. orcid.org/0000-0001-5550-4108. E-mail: 21931092@zju.edu.cn
服务　　
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章　　
闫禹铭
何剑锋
李殊昭
于慧敏
王锋
任周丰

引用本文:

闫禹铭,何剑锋,李殊昭,于慧敏,王锋,任周丰. 现实场景下行人身份识别[J]. 浙江大学学报(工学版), 2021, 55(11): 2022-2032.

Yu-ming YAN,Jian-feng HE,Shu-zhao LI,Hui-min YU,Feng WANG,Zhou-feng REN. Identity recognition under real scenes. Journal of ZheJiang University (Engineering Science), 2021, 55(11): 2022-2032.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2021.11.002    或    https://www.zjujournals.com/eng/CN/Y2021/V55/I11/2022

图 1  身份识别系统总体结构示意图
图 2  行人重识别模型
图 3  改进的衣着信息消除模块
图 4  改进的衣着信息消除模块中的注意力模块
图 5  人脸识别模型
指标 A/%
Rank-1 47.80
Rank-5 65.40
Rank-10 75.60
正样本正确划分率 82.50
无关样本正确划分率 99.98
表 1  行人重识别模型性能
模型 mAP Rank-1
ResNet-50（Baseline） 10.7 13.3
Baseline+CESD3 26.7 44.4
Baseline+ICESD3 26.9 45.6
Baseline+CESD4 26.6 44.5
Baseline+ICESD4 26.9 45.6
Baseline+CESD3+ CESD4 27.0 46.7
Baseline+ICESD3+ ICESD4 27.7 47.8
表 2  行人重识别模型组成元件分析
模型 mAP Rank-1
ResNet-50[14] 10.7 13.3
SENet-50[24] 16.5 20.5
HACNN[17] 21.6 39.9
MuDeep[16] 22.7 40.2
PCB[11] 24.3 43.6
本研究所提模型 27.7 47.8
表 3  行人重识别性能比较
数据样本 数据样本 Ac/%
正样本 总正样本 58
带人脸正样本 69
不带人脸正样 44
无关样本 总无关样本 97
带人脸无关样本 97
不带人脸无关样本 97
总样本 总样本 89
带人脸数据 92
不带人脸数据 86
表 4  测试数据结果的统计数据
模型 P/M FLOPs/G
行人重识别模型 46.6 6.0
ResNet-50[14]（行人） 24.0 4.1
人脸识别模型 24.0 1.6
ResNet-50[14]（人脸） 24.0 1.6
表 5  深度模型的参数量和FLOPs
数据类型 数据样本 t/s1)
1）注：时间为10次测试平均值.
包含人脸数据 系统运行整体时间 4.69
导入模型时间 3.90
运算时间 0.79
不包含人脸数据 系统运行整体时间 4.52
导入模型时间 3.94
运算时间 0.58
表 6  识别系统计算耗时分析
1 罗浩, 姜伟, 范星, 等 基于深度学习的行人重识别研究进展[J]. 自动化学报, 2019, 45 (11): 2032- 2049
LUO Hao, JIANG Wei, FAN Xing, et al A survey on deep learning based person re-identification[J]. Acta Automatica Sinica, 2019, 45 (11): 2032- 2049
2 李幼蛟, 卓力, 张菁, 等 行人再识别技术综述[J]. 自动化学报, 2018, 44 (9): 1554- 1568
LI You-jiao, ZHUO Li, ZHANG Jing, et al A survey of person re-identification[J]. Acta Automatica Sinica, 2018, 44 (9): 1554- 1568
3 QIAN X L, WANG W X, ZHANG L, et al. Long-term cloth-changing person re-identification [EB/OL]. [2020-05-26]. https://arxiv.org/abs/2005.12633.
4 JIN X, LAN C, ZENG W, et al. Style normalization and restitution for generalizable person re-identification[C]// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 3140-3149.
5 WEI L H, ZHANG S L, GAO W, et al. Person transfer GAN to bridge domain gap for person re-identification [C]// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 79-88.
6 KHATUN A, DENMAN S, SRIDHARAN, S, et al. End-to-end domain adaptive attention network for cross-domain person re-identification [EB/OL]. [2020-05-07]. https://arxiv.org/abs/2005.03222.
7 ZHAI Y P, LU S J. AD-Cluster: augmented discriminative clustering for domain adaptive person re-identification [C]// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 9018-9027.
8 SCHROFF F, KALENICHENKO D, PHILBIN J. FaceNet: a unified embedding for face recognition and clustering [C]// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 815-823.
9 WEN Y D, ZHANG K P, LI Z F. A discriminative feature learning approach for deep face recognition [C]// Proceedings of the 2016 European Conference on Computer Vision. Amsterdam: Springer, 2016: 499-515.
10 ZHANG J, LIU L, XU C, et al. Hierarchical and efficient learning for person re-identification [EB/OL]. [2020-05-18]. https://arxiv.org/abs/2005.08812.
11 SUN Y, ZHENG L, YANG Y, et al. Beyond part models: person retrieval with refined part pooling (and a strong convolutional baseline) [C]// Proceedings of the 2016 European Conference on Computer Vision. Munich: Springer, 2018: 3387–3396.
12 LIAO S C, HU Y, ZHU X Y, et al. Person re-identification by local maximal occurrence representation and metric learning [C]// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 2197−2206.
13 TAGORE N K, SINGH A, MANCHE S, et al. Deep learning based person re-identification [EB/OL]. [2020-05-07]. https://arxiv.org/abs/2005.03293.
14 HE K M, ZHANG X, REN S, et al. Deep residual learning for image recognition [C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
15 ULYANOV D, VEDALDI A, LEMPITSKY V. Instance normalization: the missing ingredient for fast stylization [EB/OL]. [2016-06-27]. https://arxiv.org/abs/1607.08022.
16 QIAN X L, FU Y, XIANG T, et al Leader-based multi-scale attention deep architecture for person re-identification[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42 (2): 371- 385
doi: 10.1109/TPAMI.2019.2928294
17 LI W, ZHU X, GONG S. Harmonious attention network for person re-identification [C]// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 2285–2294.
18 CHEN B, DENG W, HU J. Mixed high-order attention network for person re-identification [C]// Proceedings of the 2019 IEEE International Conference on Computer Vision. Seoul: IEEE, 2019: 371-381.
19 周勇, 王瀚正, 赵佳琦, 等. 基于可解释注意力部件模型的行人重识别方法[EB/OL]. [2020-08-23]. https://kns.cnki.net/kcms/detail/detail.aspx?dbcode=CAPJ&dbname=CAPJLAST&filename=MOTO20201105000&uniplatform=NZKPT&v=JCreBvAGjWgg9eTStxOX1MHBH7tyLrog82SWoZ97XdB1Id9qWj877hYBzM9evYAl.
ZHOU Yong, WANG Han-zheng, ZHAO Jia-qi, et al. Interpretable Attention Part Model for Person Re-identification [EB/OL]. [2020-08-23]. https:/kns.cnki.net/kcms/detail/detail.aspx?dbcode=CAPJ&dbname=CAPJLAST&filename=MOTO20201105000&uniplatform=NZKPT&v=JCreBvAGjWgg9eTStxOX1MHBH7tyLrog82SWoZ97XdB1Id9qWj877hYBzM9evYAl.
20 ZHANG K, ZHANG Z, LI Z, et al Joint face detection and alignment using multitask cascaded convolutional networks[J]. IEEE Signal Processing Letters, 2016, 23: 1499- 1503
doi: 10.1109/LSP.2016.2603342
21 JIN X., LAN C, ZENG W, et al. Global distance distributions separation for unsupervised person re-identification [EB/OL]. [2020-06-01]. https://arxiv.org/pdf/2006.00752.pdf.
22 LIN Y T, XIE L X. Unsupervised person re-identification via softened similarity learning [C]// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 3387-3396.
23 ZHENG L, SHEN L, TIAN L, et al. Scalable person re-identification: a benchmark [C]// Proceedings of the 2015 IEEE International Conference on Computer Vision. Santiago: IEEE, 2015: 1116–1124.
[1] 郭梦丽, 达飞鹏, 邓星, 盖绍彦. 基于关键点和局部特征的三维人脸识别[J]. 浙江大学学报(工学版), 2017, 51(3): 584-589.
[2] 王骕,胡浩基,于慧敏,DAMPER R I. 基于数字水印的人脸与声纹融合识别算法[J]. 浙江大学学报(工学版), 2015, 49(1): 6-14.
[3] 杨冰,王小华,杨鑫,黄孝喜. 基于HOG金字塔人脸识别方法[J]. 浙江大学学报(工学版), 2014, 48(9): 1564-1569.
[4] 张大尉, 朱善安. 基于核邻域保持判别嵌入的人脸识别[J]. J4, 2011, 45(10): 1842-1847.
[5] 赵武锋, 沈海斌, 严晓浪. 直接LDA在人脸识别中的鉴别力分析[J]. J4, 2010, 44(8): 1479-1483.
[6] 付晓峰,韦巍. 大型人脸库上基于局部二元模式直方图映射的快速识别[J]. J4, 2010, 44(1): 136-140.
[7] 孔万增 朱善安. 基于正交补空间的人脸识别[J]. J4, 2008, 42(4): 571-573.
[8] 厉小润 赵光宙 赵辽英. 改进的核直接Fisher描述分析与人脸识别[J]. J4, 2008, 42(4): 583-589.
[9] 孔万增 朱善安. 基于鉴别矢量角嵌入的人脸识别[J]. J4, 2008, 42(11): 1889-1893.
[10] 祝磊 朱善安. KSLPP：新的人脸识别算法[J]. J4, 2007, 41(7): 1066-1069.
[11] 祝磊 朱善安. 基于二维广义主成分分析的人脸识别[J]. J4, 2007, 41(2): 264-267.