Please wait a minute...
浙江大学学报(工学版)
Journal of ZheJiang University (Engineering Science)  2021, Vol. 55 Issue (11): 2022-2032    DOI: 10.3785/j.issn.1008-973X.2021.11.002
    
Identity recognition under real scenes
Yu-ming YAN1(),Jian-feng HE2,Shu-zhao LI1,Hui-min YU1,*(),Feng WANG2,Zhou-feng REN2
1. College of Information Science and Electronic Engineering, Zhejiang University, Hangzhou 310007, China
2. Jiangnan Social University, Suzhou 215124, China
Download: HTML     PDF(966KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

An identity recognition system combing face recognition and person Re-identification (Re-id) was designed for identity recognition under real scenes. When the pedestrian ’s face was detected, the recognition accuracy of the system was significantly improved through the face recognition technology. Improved cloth-elimination shape-distillation (ICESD) module was used to solve the problem of cloth-changing of pedestrians. In a real scene with a large number of unknown pedestrians, the recognition accuracy is significantly improved through the threshold filtering recognition process of identifying the known pedestrian first and then the specific pedestrian category. The application tests and comparative tests in real scenes demonstrate that the proposed Re-id method achieves significant performance improvement compared with mainstream deep learning methods under cloth-changing scenarios, and face recognition technology can also significantly improve the overall recognition accuracy. In the meantime, the real-time analysis of the system shows that the system can be deployed under real scenes.

Key words： face recognition      cloth-changing person Re-identification      real scene      identity recognition      threshold filtering     
Received: 22 March 2021      Published: 05 November 2021
CLC:  TP 183  
Fund:  之江实验室开放基金资助项目（2019KD0AB01）
Corresponding Authors: Hui-min YU     E-mail: 21931092@zju.edu.cn;yhm2005@zju.edu.cn
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Yu-ming YAN
Jian-feng HE
Shu-zhao LI
Hui-min YU
Feng WANG
Zhou-feng REN
Cite this article:

Yu-ming YAN,Jian-feng HE,Shu-zhao LI,Hui-min YU,Feng WANG,Zhou-feng REN. Identity recognition under real scenes. Journal of ZheJiang University (Engineering Science), 2021, 55(11): 2022-2032.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2021.11.002     OR     https://www.zjujournals.com/eng/Y2021/V55/I11/2022


现实场景下行人身份识别

为了实现现实场景下的行人身份识别，结合人脸识别和行人重识别（Re-id）技术设计行人身份识别系统. 人脸识别技术能够在检测到行人的人脸时提高行人身份识别的精度，行人重识别通过改进的衣着信息消除（ICESD）模块来解决现实场景下的行人换装问题. 在存在大量未知行人的现实场景下，通过先识别是否为已知行人，然后识别具体行人类别的阈值过滤识别过程显著提升准度. 现实场景的应用测试和比较试验表明，相较于主流深度学习方法，在换装场景下，行人重识别技术能实现性能的显著提升，人脸识别也能显著提升整体识别准度. 系统的实时性分析表明，行人身份系统可以布置在现实场景下.


关键词： 人脸识别,  换装行人重识别,  现实场景,  行人身份识别,  阈值过滤 
Fig.1 Overall architecture of identity recognition system
Fig.2 Person re-identification model
Fig.3 Improved cloth-elimination shape-distillation module
Fig.4 Attention module of improved cloth-elimination shape-distillation module
Fig.5 Face recognition model
指标 A/%
Rank-1 47.80
Rank-5 65.40
Rank-10 75.60
正样本正确划分率 82.50
无关样本正确划分率 99.98
Tab.1 Performance of person Re-id model
模型 mAP Rank-1
ResNet-50（Baseline） 10.7 13.3
Baseline+CESD3 26.7 44.4
Baseline+ICESD3 26.9 45.6
Baseline+CESD4 26.6 44.5
Baseline+ICESD4 26.9 45.6
Baseline+CESD3+ CESD4 27.0 46.7
Baseline+ICESD3+ ICESD4 27.7 47.8
Tab.2 Component analysis of Re-id model
模型 mAP Rank-1
ResNet-50[14] 10.7 13.3
SENet-50[24] 16.5 20.5
HACNN[17] 21.6 39.9
MuDeep[16] 22.7 40.2
PCB[11] 24.3 43.6
本研究所提模型 27.7 47.8
Tab.3 Comparison of person Re-id
数据样本 数据样本 Ac/%
正样本 总正样本 58
带人脸正样本 69
不带人脸正样 44
无关样本 总无关样本 97
带人脸无关样本 97
不带人脸无关样本 97
总样本 总样本 89
带人脸数据 92
不带人脸数据 86
Tab.4 Statisticoftestdataoutput
模型 P/M FLOPs/G
行人重识别模型 46.6 6.0
ResNet-50[14]（行人） 24.0 4.1
人脸识别模型 24.0 1.6
ResNet-50[14]（人脸） 24.0 1.6
Tab.5 Params and FLOPs of deep model
数据类型 数据样本 t/s1)
1）注：时间为10次测试平均值.
包含人脸数据 系统运行整体时间 4.69
导入模型时间 3.90
运算时间 0.79
不包含人脸数据 系统运行整体时间 4.52
导入模型时间 3.94
运算时间 0.58
Tab.6 Computing time of recognition system
[1]   罗浩, 姜伟, 范星, 等 基于深度学习的行人重识别研究进展[J]. 自动化学报, 2019, 45 (11): 2032- 2049
LUO Hao, JIANG Wei, FAN Xing, et al A survey on deep learning based person re-identification[J]. Acta Automatica Sinica, 2019, 45 (11): 2032- 2049
[2]   李幼蛟, 卓力, 张菁, 等 行人再识别技术综述[J]. 自动化学报, 2018, 44 (9): 1554- 1568
LI You-jiao, ZHUO Li, ZHANG Jing, et al A survey of person re-identification[J]. Acta Automatica Sinica, 2018, 44 (9): 1554- 1568
[3]   QIAN X L, WANG W X, ZHANG L, et al. Long-term cloth-changing person re-identification [EB/OL]. [2020-05-26]. https://arxiv.org/abs/2005.12633.
[4]   JIN X, LAN C, ZENG W, et al. Style normalization and restitution for generalizable person re-identification[C]// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 3140-3149.
[5]   WEI L H, ZHANG S L, GAO W, et al. Person transfer GAN to bridge domain gap for person re-identification [C]// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 79-88.
[6]   KHATUN A, DENMAN S, SRIDHARAN, S, et al. End-to-end domain adaptive attention network for cross-domain person re-identification [EB/OL]. [2020-05-07]. https://arxiv.org/abs/2005.03222.
[7]   ZHAI Y P, LU S J. AD-Cluster: augmented discriminative clustering for domain adaptive person re-identification [C]// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 9018-9027.
[8]   SCHROFF F, KALENICHENKO D, PHILBIN J. FaceNet: a unified embedding for face recognition and clustering [C]// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 815-823.
[9]   WEN Y D, ZHANG K P, LI Z F. A discriminative feature learning approach for deep face recognition [C]// Proceedings of the 2016 European Conference on Computer Vision. Amsterdam: Springer, 2016: 499-515.
[10]   ZHANG J, LIU L, XU C, et al. Hierarchical and efficient learning for person re-identification [EB/OL]. [2020-05-18]. https://arxiv.org/abs/2005.08812.
[11]   SUN Y, ZHENG L, YANG Y, et al. Beyond part models: person retrieval with refined part pooling (and a strong convolutional baseline) [C]// Proceedings of the 2016 European Conference on Computer Vision. Munich: Springer, 2018: 3387–3396.
[12]   LIAO S C, HU Y, ZHU X Y, et al. Person re-identification by local maximal occurrence representation and metric learning [C]// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 2197−2206.
[13]   TAGORE N K, SINGH A, MANCHE S, et al. Deep learning based person re-identification [EB/OL]. [2020-05-07]. https://arxiv.org/abs/2005.03293.
[14]   HE K M, ZHANG X, REN S, et al. Deep residual learning for image recognition [C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
[15]   ULYANOV D, VEDALDI A, LEMPITSKY V. Instance normalization: the missing ingredient for fast stylization [EB/OL]. [2016-06-27]. https://arxiv.org/abs/1607.08022.
[16]   QIAN X L, FU Y, XIANG T, et al Leader-based multi-scale attention deep architecture for person re-identification[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42 (2): 371- 385
doi: 10.1109/TPAMI.2019.2928294
[17]   LI W, ZHU X, GONG S. Harmonious attention network for person re-identification [C]// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 2285–2294.
[18]   CHEN B, DENG W, HU J. Mixed high-order attention network for person re-identification [C]// Proceedings of the 2019 IEEE International Conference on Computer Vision. Seoul: IEEE, 2019: 371-381.
[19]   周勇, 王瀚正, 赵佳琦, 等. 基于可解释注意力部件模型的行人重识别方法[EB/OL]. [2020-08-23]. https://kns.cnki.net/kcms/detail/detail.aspx?dbcode=CAPJ&dbname=CAPJLAST&filename=MOTO20201105000&uniplatform=NZKPT&v=JCreBvAGjWgg9eTStxOX1MHBH7tyLrog82SWoZ97XdB1Id9qWj877hYBzM9evYAl.
ZHOU Yong, WANG Han-zheng, ZHAO Jia-qi, et al. Interpretable Attention Part Model for Person Re-identification [EB/OL]. [2020-08-23]. https:/kns.cnki.net/kcms/detail/detail.aspx?dbcode=CAPJ&dbname=CAPJLAST&filename=MOTO20201105000&uniplatform=NZKPT&v=JCreBvAGjWgg9eTStxOX1MHBH7tyLrog82SWoZ97XdB1Id9qWj877hYBzM9evYAl.
[20]   ZHANG K, ZHANG Z, LI Z, et al Joint face detection and alignment using multitask cascaded convolutional networks[J]. IEEE Signal Processing Letters, 2016, 23: 1499- 1503
doi: 10.1109/LSP.2016.2603342
[21]   JIN X., LAN C, ZENG W, et al. Global distance distributions separation for unsupervised person re-identification [EB/OL]. [2020-06-01]. https://arxiv.org/pdf/2006.00752.pdf.
[22]   LIN Y T, XIE L X. Unsupervised person re-identification via softened similarity learning [C]// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 3387-3396.
[23]   ZHENG L, SHEN L, TIAN L, et al. Scalable person re-identification: a benchmark [C]// Proceedings of the 2015 IEEE International Conference on Computer Vision. Santiago: IEEE, 2015: 1116–1124.
[1] Jin GAO,Jing-hai GONG,Jun HE. Evacuation from offshore platform based on multi-velocity cellular automata[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(9): 1764-1771.
[2] GUO Meng-li, DA Fei-peng, DENG Xing, GAI Shao-yan. 3D face recognition based on keypoints and local feature[J]. Journal of ZheJiang University (Engineering Science), 2017, 51(3): 584-589.
[3] WANG Su, HU Hao-ji, YU Hui-min, DAMPER R I. Augmenting remote multimodal person verification by embedding voice characteristics into face images[J]. Journal of ZheJiang University (Engineering Science), 2015, 49(1): 6-14.
[4] YANG Bing, WANG Xiao-hua, YANG Xin, HUANG Xiao-xi. Face recognition method based on HOG pyramid[J]. Journal of ZheJiang University (Engineering Science), 2014, 48(9): 1564-1569.
[5] ZHANG Da-wei, ZHU Shan-an. Face recognition based kernel neighborhood preserving
discriminant embedding[J]. Journal of ZheJiang University (Engineering Science), 2011, 45(10): 1842-1847.
[6] DIAO Wu-Feng, CHEN Hai-Bin, YAN Xiao-Lang. Discriminative analysis of direct linear discriminant
analysis for face recognition[J]. Journal of ZheJiang University (Engineering Science), 2010, 44(8): 1479-1483.
[7] FU Xiao-feng, WEI Wei. Local binary pattern histogram projection based fast recognition on large-scale face database[J]. Journal of ZheJiang University (Engineering Science), 2010, 44(1): 136-140.