Please wait a minute...
浙江大学学报(工学版)
浙江大学学报(工学版)  2022, Vol. 56 Issue (7): 1416-1424    DOI: 10.3785/j.issn.1008-973X.2022.07.017
土木工程、水利工程、交通工程     
基于能量模型的行人与车辆再识别方法
张师林(),郭红南,刘轩
北方工业大学 城市道路交通智能控制技术北京市重点实验室，北京 100144
Person and vehicle re-identification based on energy model
Shi-lin ZHANG(),Hong-nan GUO,Xuan LIU
Beijing Key Laboratory of Traffic Intelligent Control, North China University of Technology, Beijing 100144, China
 全文: PDF(1265 KB)   HTML
摘要：

为了解决行人再识别以及车辆再识别算法中网络训练过程对计算资源的消耗过大且准确率较低的问题，提出基于能量模型的目标分类和度量学习方法. 利用样本特征空间中同类样本的低能量分布特性, 设计对比能量损失函数，形式上表达为训练样本在真实目标类别上的损失函数响应和非目标类别上的响应之差，可以更准确地增大目标响应，抑制非目标响应, 提高了分类准确率，使得同类样本特征更聚集、异类样本特征更远离. 在多个行人再识别和车辆再识别数据集上的测试结果显示, 相对于Soft-max和Triplet混合损失函数, 利用能量模型可以提升网络训练效率，提高目标再识别准确率.
关键词： 车辆再识别;  能量模型;  行人再识别;  损失函数;  三元组损失    
Abstract:

An energy-based object detection and metric learning method was proposed in order to solve the intensive computational cost and low accuracy issues in the training process of person re-identification and vehicle re-identification (re-ID) algorithms. A contrastive energy-based loss was designed based on the low energy characteristic of the same samples in the feature space, which took the form of the difference between samples’ true target response and non-target response of the training loss. The target response can be increased more accurately, and the non-target response can be suppressed. The classification accuracy can be improved, and the features within the same categories can stay more compact while different identities keep a distance away. Experiments on several person re-ID and vehicle re-ID databases showed that the efficiency of the training process was improved and the object re-ID accuracy was enhanced compared to the fused loss of Soft-max and Triplet.
Key words: vehicle re-identification    energy model    person re-identification    loss function    Triplet loss
收稿日期: 2021-07-06 出版日期: 2022-07-26
CLC:  TP 391  
基金资助: 国家自然科学基金资助项目（61403002）；北方工业大学优秀青年人才资助项目
作者简介: 张师林（1980—），男，副教授，从事人工智能的研究. orcid.org/0000-0003-3034-1538. E-mail： zhangshilin@126.com
服务　　
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章　　
张师林
郭红南
刘轩

引用本文:

张师林,郭红南,刘轩. 基于能量模型的行人与车辆再识别方法[J]. 浙江大学学报(工学版), 2022, 56(7): 1416-1424.

Shi-lin ZHANG,Hong-nan GUO,Xuan LIU. Person and vehicle re-identification based on energy model. Journal of ZheJiang University (Engineering Science), 2022, 56(7): 1416-1424.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2022.07.017    或    https://www.zjujournals.com/eng/CN/Y2022/V56/I7/1416

图 1  基于对比能量模型的网络训练过程整体框架
T α β Rank1/% mAP/%
1 1 1 94.2 84.0
10?3 1 1 94.4 84.5
10?6 1 1 94.5 85.1
10?6 0.5 0.5 94.8 86.1
10?6 0.5 0.3 94.8 86.7
10?6 0.3 0.5 95.1 87.8
10?6 0.3 0.3 95.2 88.1
10?6 0.1 0.3 95.0 87.6
10?6 0.1 0.1 94.8 86.5
表 1  对比能量损失中各参数在Market1501数据集上对行人再识别性能的影响
图 2  随机采样的车辆样本的特征分布
%
网络 损失函数 Rank1 mAP
OSNet Soft-max 93.2 83.5
OSNet AM-Soft-max 94.1 84.5
OSNet Arc-Soft-max 94.3 84.9
OSNet Soft-max+Center 94.8 85.1
OSNet Soft-max+Triplet 95.3 87.5
OSNet Energy-Loss 95.7 88.1
ResNet50 Soft-max 93.6 83.8
ResNet50 AM-Soft-max 94.4 84.7
ResNet50 Arc-Soft-max 94.7 85.2
ResNet50 Soft-max+Center 94.9 86.2
ResNet50 Soft-max+Triplet 95.4 88.1
ResNet50 Energy-Loss 95.9 88.5
ResNet50-NL Soft-max 94.3 84.2
ResNet50-NL AM-Soft-max 94.5 85.3
ResNet50-NL Arc-Soft-max 95.1 85.9
ResNet50-NL Soft-max+Center 95.2 86.8
ResNet50-NL Soft-max+Triplet 95.6 88.2
ResNet50-NL Energy-Loss 96.1 88.7
表 2  不同损失函数在不同基础网络上的性能比较
图 3  网络模型在对比能量损失和混合损失条件下所生成的图像特征图
%
方法 Market1501 DukeMTMC-ReID MSMT
Rank1 mAP Rank1 mAP Rank1 mAP
Camstyle[19] 88.1 68.7 75.3 53.5 — —
PN-GAN[20] 89.4 72.6 73.6 53.2 — —
MGN[21] 95.7 86.9 88.7 78.4 — —
Pyramid[22] 95.7 88.2 89.0 79.0 — —
ABD-Net[23] 95.6 88.3 88.3 78.6 — —
PCB[24] 93.8 81.6 83.3 69.2 68.2 40.4
SPReID[25] 92.5 81.3 84.4 71.0 — —
MaskReID[26] 90.0 75.3 78.8 61.9 — —
SCPNet[27] 91.2 75.2 80.3 62.6 — —
HA-CNN[28] 91.2 75.7 80.5 63.8 — —
SVDNet[29] 82.3 62.1 76.7 56.8 — —
TransReID[30] 95.2 89.5 91.1 82.1 86.20 69.4
Energy-Loss 95.9 89.9 92.3 83.5 85.52 70.9
表 3  在3个行人再识别数据集上与同类方法的Rank1 和mAP指标对比
%
方法 VehicleID Small VehicleID Medium VehicleID Large
Rank1 mAP Rank1 mAP Rank1 mAP
CLVR[32] 62.00 — 56.10 — 50.60 —
VANet[33] 88.12 — 83.17 — 80.45 —
RAM[34] 75.20 — 72.30 — 67.70 —
ABLN[35] 52.63 — — — — —
VAMI[36] 63.12 — 52.87 — 47.34 —
NuFACT[37] 48.90 — 43.64 — 38.63 —
AAVER[38] 74.69 — 68.62 — 63.54 —
QD-DLF[39] 72.32 76.54 70.66 74.63 64.14 68.41
Part-Reg[40] 78.40 61.50 75.00 — 74.20 —
GSTE[41] 75.90 75.40 74.80 74.30 74.00 72.40
Energy-Loss 89.75 85.82 84.58 81.35 81.15 77.68
表 4  在VehicleID 数据集上与最好方法的Rank1 和mAP指标对比
图 4  能量模型下行人再识别结果的可视化
1 LIU W, WEN Y, YU Z, et al. Sphere face: deep hypersphere embedding for face recognition [C]// Proceedings of the Computer Vision and Pattern Recognition. Hawaii: IEEE, 2017: 6738-6746.
2 DENG J K, GUO J, XUE N N, et al. Arcface: additive angular margin loss for deep face recognition [C]// Proceedings of the Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 4685-4694.
3 WANG F, CHENG L, LIU W Additive margin Soft-max for face verification[J]. IEEE Signal Processing Letters, 2018, 25 (7): 926- 930
doi: 10.1109/LSP.2018.2822810
4 JIQUAN N, CHEN Z H, PANG W, et al. Learning deep energy models [C]// Proceedings of the 28th International Conference on Machine Learning, Washington: Omnipress, 2011: 1105-1112.
5 TAESUP K , YOSHUA B. Deep directed generative models with energy-based probability estimation [C]// Proceedings of the European Conference of Computer Vision. Amsterdam: Springer, 2016: 123-130.
6 YANN L, SUMIT C, RAIA H. A tutorial on energy-based learning [M]// Predicting structured data. Boston: MIT Press, 2006.
7 RITHESH K, ANIRUDH G, AARON C, et al. Maximum entropy generators for energy based models [C]// Proceedings of the International Conference on Computer Vision. Seoul: IEEE, 2019: 1701-1711.
8 LIU W T, WANG X Y, OWENS J. Energy-based out-of-distribution detection [C]// Proceedings of the Neural Information Processing System. Canada: IEEE, 2020: 112-123.
9 ZHOU K Y, YANG Y X, CAVALLARO A. Omni-scale feature learning for person re-identification [C]// Proceedings of the International Conference on Computer Vision. Seoul: IEEE, 2019: 3701-3711.
10 ZHENG L, SHEN L Y, TIAN L. Scalable person re-identification: a benchmark [C]// Proceedings of the Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 1116-1124.
11 RISTANI E, SOLERA F, ZOU R, et al. Performance measures and a data set for multi-target, multi-camera tracking [C]// Proceedings of the European Conference of Computer Vision. Amsterdam: Springer, 2016: 17-35.
12 WEI L H, ZHANG S L, GAO W, et al. Person transfer gan to bridge domain gap for person re-identification [C]// Proceedings of the Computer Vision and Pattern Recognition. Utah: IEEE, 2018: 79-88.
13 LIU X C, LIU W, MEI T. A deep learning-based approach to progressive vehicle re-identification for urban surveillance [C]// Proceedings of the European Conference of Computer Vision. Amsterdam: Springer, 2016: 123-130.
14 LIU H Y, TIAN Y H, WANG Y W. Deep relative distance learning: tell the difference between similar vehicles [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Nevada: IEEE, 2016: 2167-2175.
15 HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Nevada: IEEE, 2016: 116-124.
16 WANG X, GIRSHICK R, GUPTA A, et al. Non-local neural networks [C]// Proceedings of the Computer Vision and Pattern Recognition. Utah: IEEE, 2018: 7794-7803.
17 WEN Y, ZHANG K, LI Z. A discriminative feature learning approach for deep face recognition [C]// Proceedings of the European Conference of Computer Vision. Amsterdam: Springer, 2016: 23-30.
18 ALEXANDER H, LUCAS B, BASTIAN L. In defense of the triplet loss for person re-identification [C]// Proceedings of the International Conference on Computer Vision. Seoul: Springer, 2018: 1132-1139.
19 ZHONG Z, ZHENG L, ZHENG Z D, et al Camstyle: a novel data augmentation method for person re-identification[J]. IEEE Transactions on Image Processing, 2019, 28 (3): 1176- 1190
doi: 10.1109/TIP.2018.2874313
20 QIAN X L, FU Y W, XIANG T, et al. Pose-normalized image generation for person re-identification [C]// Proceedings of the European Conference of Computer Vision. Munich: Springer, 2018: 1123-1132.
21 WANG G S, YUAN Y F, CHEN X, et al. Learning discriminative features with multiple granularities for person re-identification [C]// Proceedings of the ACM Multimedia Conference on Multimedia Conference. Seoul: ACM, 2018: 1123-1132.
22 ZHENG F, DENG C, SUN X, et al. Pyramidal person re-identification via multi-loss dynamic training [C]// Proceedings of the Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 134-143.
23 CHEN T S, XU M X, HUI X L, et al. Learning semantic-specific graph representation for multi-label image recognition [C]// Proceedings of the International Conference on Computer Vision. Seoul: Springer, 2019: 2132-2139.
24 SUN Y F, ZHENG L, YANG Y. Beyond part models: person retrieval with refined part pooling and a strong convolutional baseline [C]// Proceedings of the European Conference of Computer Vision. Munich: Springer, 2018: 25-32.
25 KAKAYEH M M, BASRARN E. Human semantic parsing for person re-identification [C]// Proceedings of the Computer Vision and Pattern Recognition. Utah: IEEE, 2018: 99--107.
26 LEI Q, JING H, LEI W, et al. Maskreid: a mask based deep ranking neural network for person re-identification [C]// Proceedings of the International Conference of Multimedia Exposition. Shanghai: IEEE, 2019: 1138-1145.
27 FAN X, LUO H, ZHANG X, et al. SCPNet: spatial-channel parallelism network for joint holistic and partial person re-identification [C]// Proceedings of the Asian Conference of Computer Vision. Kyoto: IEEE, 2019: 2351-2359.
28 LI W, ZHU X T, GONG S G. Harmonious attention network for person re-identification [C]// Proceedings of the Computer Vision and Pattern Recognition. Utah: IEEE, 2018: 1324—1332.
29 SUN Y F, ZHENG L, DENG W J, et al. SVDNet for pedestrian retrieval[C]// Proceedings of the Computer Vision and Pattern Recognition. Utah: IEEE, 2018: 99-107.
30 HE S, LUO H, WANG P C, et al. TransReID: transformer-based object re-identification [C]// Proceedings of the Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 151-159.
31 ZHONG Z, ZHENG L, CAO D L, et al. Re-ranking person re-identification with k-reciprocal encoding [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Hawaii: IEEE, 2017: 345-352.
32 LIU X C, LIU W, MEI T, et al. A deep learning-based approach to progressive vehicle re-identification for urban surveillance [C]// Proceedings of the European Conference of Computer Vision. Amsterdam: Springer, 2016: 123-130.
33 CHU R H, SUN Y F, LI Y D, et al. Vehicle re-identification with viewpoint aware metric learning [C]// Proceedings of the International Conference on Computer Vision. Seoul: Springer, 2019: 1132-1139.
34 LIU X B, ZHANG S L, HUANG Q M, et al. Ram: a region-aware deep model for vehicle re-identification [C]// Proceedings of the International Conference of Multimedia Exposition. San Diego: IEEE, 2018: 138-145.
35 ZHOU Y, SHAO L. Vehicle re-identification by adversarial bi-directional LSTM network [C]// Proceedings of the IEEE Winter Conference on Applications of Computer Vision. Salt Lake City: IEEE, 2018: 1123-1132.
36 ZHOU Y, SHAO L. Viewpoint-aware attentive multi-view inference for vehicle re-identification [C]// Proceedings of the Computer Vision and Pattern Recognition. Utah: IEEE, 2018: 324-332.
37 LIU X C, LIU W, MEI T, et al PROVID: progressive and multimodal vehicle reidentification for large-scale urban surveillance[J]. IEEE Transactions on Multimedia, 2018, 20 (3): 645- 658
doi: 10.1109/TMM.2017.2751966
38 KHORRAMSHAHI P, KUMAR A, PERI N, et al. A dual-path model with adaptive attention for vehicle re-identification [C]// Proceedings of the International Conference on Computer Vision. Seoul: Springer, 2019: 132-139.
39 ZHU J Q, ZENG H Q, HUANG J C, et al Vehicle re-identification using quadruple directional deep learning features[J]. IEEE Transactions on Intelligent Transportation Systems, 2019, 21 (1): 1- 11
40 HE B, LI J, ZHAO Y, et al. Part-regularized near-duplicate vehicle re-identification [C]// Proceedings of the Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 154-163.
[1] 张师林,马思明,顾子谦. 基于大边距度量学习的车辆再识别方法[J]. 浙江大学学报(工学版), 2021, 55(5): 948-956.
[2] 刘芳,汪震,刘睿迪,王锴. 基于组合损失函数的BP神经网络风力发电短期预测方法[J]. 浙江大学学报(工学版), 2021, 55(3): 594-600.