Please wait a minute...
浙江大学学报(工学版)
Journal of ZheJiang University (Engineering Science)  2022, Vol. 56 Issue (7): 1416-1424    DOI: 10.3785/j.issn.1008-973X.2022.07.017
    
Person and vehicle re-identification based on energy model
Shi-lin ZHANG(),Hong-nan GUO,Xuan LIU
Beijing Key Laboratory of Traffic Intelligent Control, North China University of Technology, Beijing 100144, China
Download: HTML     PDF(1265KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

An energy-based object detection and metric learning method was proposed in order to solve the intensive computational cost and low accuracy issues in the training process of person re-identification and vehicle re-identification (re-ID) algorithms. A contrastive energy-based loss was designed based on the low energy characteristic of the same samples in the feature space, which took the form of the difference between samples’ true target response and non-target response of the training loss. The target response can be increased more accurately, and the non-target response can be suppressed. The classification accuracy can be improved, and the features within the same categories can stay more compact while different identities keep a distance away. Experiments on several person re-ID and vehicle re-ID databases showed that the efficiency of the training process was improved and the object re-ID accuracy was enhanced compared to the fused loss of Soft-max and Triplet.

Key words： vehicle re-identification      energy model      person re-identification      loss function      Triplet loss     
Received: 06 July 2021      Published: 26 July 2022
CLC:  TP 391  
Fund:  国家自然科学基金资助项目（61403002）；北方工业大学优秀青年人才资助项目
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Shi-lin ZHANG
Hong-nan GUO
Xuan LIU
Cite this article:

Shi-lin ZHANG,Hong-nan GUO,Xuan LIU. Person and vehicle re-identification based on energy model. Journal of ZheJiang University (Engineering Science), 2022, 56(7): 1416-1424.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2022.07.017     OR     https://www.zjujournals.com/eng/Y2022/V56/I7/1416


基于能量模型的行人与车辆再识别方法

为了解决行人再识别以及车辆再识别算法中网络训练过程对计算资源的消耗过大且准确率较低的问题，提出基于能量模型的目标分类和度量学习方法. 利用样本特征空间中同类样本的低能量分布特性, 设计对比能量损失函数，形式上表达为训练样本在真实目标类别上的损失函数响应和非目标类别上的响应之差，可以更准确地增大目标响应，抑制非目标响应, 提高了分类准确率，使得同类样本特征更聚集、异类样本特征更远离. 在多个行人再识别和车辆再识别数据集上的测试结果显示, 相对于Soft-max和Triplet混合损失函数, 利用能量模型可以提升网络训练效率，提高目标再识别准确率.


关键词： 车辆再识别,  能量模型,  行人再识别,  损失函数,  三元组损失 
Fig.1 Overall structure of contrastive energy-based method in network training process
T α β Rank1/% mAP/%
1 1 1 94.2 84.0
10?3 1 1 94.4 84.5
10?6 1 1 94.5 85.1
10?6 0.5 0.5 94.8 86.1
10?6 0.5 0.3 94.8 86.7
10?6 0.3 0.5 95.1 87.8
10?6 0.3 0.3 95.2 88.1
10?6 0.1 0.3 95.0 87.6
10?6 0.1 0.1 94.8 86.5
Tab.1 Parameter influences on performance of person re-ID on Market1501 in relative energy loss
Fig.2 Feature distribution of randomly selected vehicles
%
网络 损失函数 Rank1 mAP
OSNet Soft-max 93.2 83.5
OSNet AM-Soft-max 94.1 84.5
OSNet Arc-Soft-max 94.3 84.9
OSNet Soft-max+Center 94.8 85.1
OSNet Soft-max+Triplet 95.3 87.5
OSNet Energy-Loss 95.7 88.1
ResNet50 Soft-max 93.6 83.8
ResNet50 AM-Soft-max 94.4 84.7
ResNet50 Arc-Soft-max 94.7 85.2
ResNet50 Soft-max+Center 94.9 86.2
ResNet50 Soft-max+Triplet 95.4 88.1
ResNet50 Energy-Loss 95.9 88.5
ResNet50-NL Soft-max 94.3 84.2
ResNet50-NL AM-Soft-max 94.5 85.3
ResNet50-NL Arc-Soft-max 95.1 85.9
ResNet50-NL Soft-max+Center 95.2 86.8
ResNet50-NL Soft-max+Triplet 95.6 88.2
ResNet50-NL Energy-Loss 96.1 88.7
Tab.2 Performance comparison of different loss functions over different networks
Fig.3 Feature maps produced by energy loss and fused loss based network
%
方法 Market1501 DukeMTMC-ReID MSMT
Rank1 mAP Rank1 mAP Rank1 mAP
Camstyle[19] 88.1 68.7 75.3 53.5 — —
PN-GAN[20] 89.4 72.6 73.6 53.2 — —
MGN[21] 95.7 86.9 88.7 78.4 — —
Pyramid[22] 95.7 88.2 89.0 79.0 — —
ABD-Net[23] 95.6 88.3 88.3 78.6 — —
PCB[24] 93.8 81.6 83.3 69.2 68.2 40.4
SPReID[25] 92.5 81.3 84.4 71.0 — —
MaskReID[26] 90.0 75.3 78.8 61.9 — —
SCPNet[27] 91.2 75.2 80.3 62.6 — —
HA-CNN[28] 91.2 75.7 80.5 63.8 — —
SVDNet[29] 82.3 62.1 76.7 56.8 — —
TransReID[30] 95.2 89.5 91.1 82.1 86.20 69.4
Energy-Loss 95.9 89.9 92.3 83.5 85.52 70.9
Tab.3 Rank1 and mAP performance comparison with state of art methods on three person re-ID datasets
%
方法 VehicleID Small VehicleID Medium VehicleID Large
Rank1 mAP Rank1 mAP Rank1 mAP
CLVR[32] 62.00 — 56.10 — 50.60 —
VANet[33] 88.12 — 83.17 — 80.45 —
RAM[34] 75.20 — 72.30 — 67.70 —
ABLN[35] 52.63 — — — — —
VAMI[36] 63.12 — 52.87 — 47.34 —
NuFACT[37] 48.90 — 43.64 — 38.63 —
AAVER[38] 74.69 — 68.62 — 63.54 —
QD-DLF[39] 72.32 76.54 70.66 74.63 64.14 68.41
Part-Reg[40] 78.40 61.50 75.00 — 74.20 —
GSTE[41] 75.90 75.40 74.80 74.30 74.00 72.40
Energy-Loss 89.75 85.82 84.58 81.35 81.15 77.68
Tab.4 Rank1 and mAP performance comparison with state of art methods on VehicleID
Fig.4 Visualization of person re-identification under energy model
[1]   LIU W, WEN Y, YU Z, et al. Sphere face: deep hypersphere embedding for face recognition [C]// Proceedings of the Computer Vision and Pattern Recognition. Hawaii: IEEE, 2017: 6738-6746.
[2]   DENG J K, GUO J, XUE N N, et al. Arcface: additive angular margin loss for deep face recognition [C]// Proceedings of the Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 4685-4694.
[3]   WANG F, CHENG L, LIU W Additive margin Soft-max for face verification[J]. IEEE Signal Processing Letters, 2018, 25 (7): 926- 930
doi: 10.1109/LSP.2018.2822810
[4]   JIQUAN N, CHEN Z H, PANG W, et al. Learning deep energy models [C]// Proceedings of the 28th International Conference on Machine Learning, Washington: Omnipress, 2011: 1105-1112.
[5]   TAESUP K , YOSHUA B. Deep directed generative models with energy-based probability estimation [C]// Proceedings of the European Conference of Computer Vision. Amsterdam: Springer, 2016: 123-130.
[6]   YANN L, SUMIT C, RAIA H. A tutorial on energy-based learning [M]// Predicting structured data. Boston: MIT Press, 2006.
[7]   RITHESH K, ANIRUDH G, AARON C, et al. Maximum entropy generators for energy based models [C]// Proceedings of the International Conference on Computer Vision. Seoul: IEEE, 2019: 1701-1711.
[8]   LIU W T, WANG X Y, OWENS J. Energy-based out-of-distribution detection [C]// Proceedings of the Neural Information Processing System. Canada: IEEE, 2020: 112-123.
[9]   ZHOU K Y, YANG Y X, CAVALLARO A. Omni-scale feature learning for person re-identification [C]// Proceedings of the International Conference on Computer Vision. Seoul: IEEE, 2019: 3701-3711.
[10]   ZHENG L, SHEN L Y, TIAN L. Scalable person re-identification: a benchmark [C]// Proceedings of the Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 1116-1124.
[11]   RISTANI E, SOLERA F, ZOU R, et al. Performance measures and a data set for multi-target, multi-camera tracking [C]// Proceedings of the European Conference of Computer Vision. Amsterdam: Springer, 2016: 17-35.
[12]   WEI L H, ZHANG S L, GAO W, et al. Person transfer gan to bridge domain gap for person re-identification [C]// Proceedings of the Computer Vision and Pattern Recognition. Utah: IEEE, 2018: 79-88.
[13]   LIU X C, LIU W, MEI T. A deep learning-based approach to progressive vehicle re-identification for urban surveillance [C]// Proceedings of the European Conference of Computer Vision. Amsterdam: Springer, 2016: 123-130.
[14]   LIU H Y, TIAN Y H, WANG Y W. Deep relative distance learning: tell the difference between similar vehicles [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Nevada: IEEE, 2016: 2167-2175.
[15]   HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Nevada: IEEE, 2016: 116-124.
[16]   WANG X, GIRSHICK R, GUPTA A, et al. Non-local neural networks [C]// Proceedings of the Computer Vision and Pattern Recognition. Utah: IEEE, 2018: 7794-7803.
[17]   WEN Y, ZHANG K, LI Z. A discriminative feature learning approach for deep face recognition [C]// Proceedings of the European Conference of Computer Vision. Amsterdam: Springer, 2016: 23-30.
[18]   ALEXANDER H, LUCAS B, BASTIAN L. In defense of the triplet loss for person re-identification [C]// Proceedings of the International Conference on Computer Vision. Seoul: Springer, 2018: 1132-1139.
[19]   ZHONG Z, ZHENG L, ZHENG Z D, et al Camstyle: a novel data augmentation method for person re-identification[J]. IEEE Transactions on Image Processing, 2019, 28 (3): 1176- 1190
doi: 10.1109/TIP.2018.2874313
[20]   QIAN X L, FU Y W, XIANG T, et al. Pose-normalized image generation for person re-identification [C]// Proceedings of the European Conference of Computer Vision. Munich: Springer, 2018: 1123-1132.
[21]   WANG G S, YUAN Y F, CHEN X, et al. Learning discriminative features with multiple granularities for person re-identification [C]// Proceedings of the ACM Multimedia Conference on Multimedia Conference. Seoul: ACM, 2018: 1123-1132.
[22]   ZHENG F, DENG C, SUN X, et al. Pyramidal person re-identification via multi-loss dynamic training [C]// Proceedings of the Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 134-143.
[23]   CHEN T S, XU M X, HUI X L, et al. Learning semantic-specific graph representation for multi-label image recognition [C]// Proceedings of the International Conference on Computer Vision. Seoul: Springer, 2019: 2132-2139.
[24]   SUN Y F, ZHENG L, YANG Y. Beyond part models: person retrieval with refined part pooling and a strong convolutional baseline [C]// Proceedings of the European Conference of Computer Vision. Munich: Springer, 2018: 25-32.
[25]   KAKAYEH M M, BASRARN E. Human semantic parsing for person re-identification [C]// Proceedings of the Computer Vision and Pattern Recognition. Utah: IEEE, 2018: 99--107.
[26]   LEI Q, JING H, LEI W, et al. Maskreid: a mask based deep ranking neural network for person re-identification [C]// Proceedings of the International Conference of Multimedia Exposition. Shanghai: IEEE, 2019: 1138-1145.
[27]   FAN X, LUO H, ZHANG X, et al. SCPNet: spatial-channel parallelism network for joint holistic and partial person re-identification [C]// Proceedings of the Asian Conference of Computer Vision. Kyoto: IEEE, 2019: 2351-2359.
[28]   LI W, ZHU X T, GONG S G. Harmonious attention network for person re-identification [C]// Proceedings of the Computer Vision and Pattern Recognition. Utah: IEEE, 2018: 1324—1332.
[29]   SUN Y F, ZHENG L, DENG W J, et al. SVDNet for pedestrian retrieval[C]// Proceedings of the Computer Vision and Pattern Recognition. Utah: IEEE, 2018: 99-107.
[30]   HE S, LUO H, WANG P C, et al. TransReID: transformer-based object re-identification [C]// Proceedings of the Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 151-159.
[31]   ZHONG Z, ZHENG L, CAO D L, et al. Re-ranking person re-identification with k-reciprocal encoding [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Hawaii: IEEE, 2017: 345-352.
[32]   LIU X C, LIU W, MEI T, et al. A deep learning-based approach to progressive vehicle re-identification for urban surveillance [C]// Proceedings of the European Conference of Computer Vision. Amsterdam: Springer, 2016: 123-130.
[33]   CHU R H, SUN Y F, LI Y D, et al. Vehicle re-identification with viewpoint aware metric learning [C]// Proceedings of the International Conference on Computer Vision. Seoul: Springer, 2019: 1132-1139.
[34]   LIU X B, ZHANG S L, HUANG Q M, et al. Ram: a region-aware deep model for vehicle re-identification [C]// Proceedings of the International Conference of Multimedia Exposition. San Diego: IEEE, 2018: 138-145.
[35]   ZHOU Y, SHAO L. Vehicle re-identification by adversarial bi-directional LSTM network [C]// Proceedings of the IEEE Winter Conference on Applications of Computer Vision. Salt Lake City: IEEE, 2018: 1123-1132.
[36]   ZHOU Y, SHAO L. Viewpoint-aware attentive multi-view inference for vehicle re-identification [C]// Proceedings of the Computer Vision and Pattern Recognition. Utah: IEEE, 2018: 324-332.
[37]   LIU X C, LIU W, MEI T, et al PROVID: progressive and multimodal vehicle reidentification for large-scale urban surveillance[J]. IEEE Transactions on Multimedia, 2018, 20 (3): 645- 658
doi: 10.1109/TMM.2017.2751966
[38]   KHORRAMSHAHI P, KUMAR A, PERI N, et al. A dual-path model with adaptive attention for vehicle re-identification [C]// Proceedings of the International Conference on Computer Vision. Seoul: Springer, 2019: 132-139.
[39]   ZHU J Q, ZENG H Q, HUANG J C, et al Vehicle re-identification using quadruple directional deep learning features[J]. IEEE Transactions on Intelligent Transportation Systems, 2019, 21 (1): 1- 11
[40]   HE B, LI J, ZHAO Y, et al. Part-regularized near-duplicate vehicle re-identification [C]// Proceedings of the Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 154-163.
[1] Shi-lin ZHANG,Si-ming MA,Zi-qian GU. Large margin metric learning based vehicle re-identification method[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(5): 948-956.
[2] Fang LIU,Zhen WANG,Rui-di LIU,Kai WANG. Short-term forecasting method of wind power generation based on BP neural network with combined loss function[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(3): 594-600.
[3] Yu-ming YAN,Jian-feng HE,Shu-zhao LI,Hui-min YU,Feng WANG,Zhou-feng REN. Identity recognition under real scenes[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(11): 2022-2032.
[4] Jin LIANG,Kun LUO,Qiang WANG,Xu-chao YANG,Jian-ren FAN,Jun-xi ZHANG. Effect of high-albedo roofs on urban heat island and air-conditioning energy consumption[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(10): 1993-2000.
[5] LIU Duan-yang , Xie Jian-ping, CAO Yan-long. Research on divisible load scheduling algorithm based on energy model[J]. Journal of ZheJiang University (Engineering Science), 2013, 47(9): 1547-1553.