Please wait a minute...
Journal of ZheJiang University (Engineering Science)  2021, Vol. 55 Issue (5): 948-956    DOI: 10.3785/j.issn.1008-973X.2021.05.015
    
Large margin metric learning based vehicle re-identification method
Shi-lin ZHANG(),Si-ming MA,Zi-qian GU
Beijing Key Laboratory of Traffic Intelligent Control, North China University of Technology, Beijing 100144, China
Download: HTML     PDF(891KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

In the present vehicle re-identification methods, it is hard to separate the samples near the decision boundaries in the embedding feature space because of the multi-view vehicle appearance. A large margin metric learning method was proposed to tackle the above problem. In the representation stage, a large margin loss function was presented to tackle the identity confusion issue among similar vehicles effectively. Meanwhile, an invader-defector sampler was adopted, which was used to find the hard samples in the training dataset to train the network specifically and speed up the training process. A kernelized re-rank method was adopted to further enhance the re-identification performance in the retrieval stage. Experiments on three common databases show that the proposed method can achieve a higher vehicle re-identification accuracy, and at the same time the time cost of training and the inference procedure are improved. Theoretical analysis and experiments also indicate that, the large margin metric learning method can mine the hard samples near the decision boundaries, and can solve the multi-view issue in the vehicle re-identification domain.



Key wordsvehicle re-identification      feature embedding      metric learning      loss function      sampling strategy      kernel function     
Received: 29 June 2020      Published: 10 June 2021
CLC:  TP 391  
Fund:  国家自然科学基金资助项目(61403002);北方工业大学优秀青年人才资助项目
Cite this article:

Shi-lin ZHANG,Si-ming MA,Zi-qian GU. Large margin metric learning based vehicle re-identification method. Journal of ZheJiang University (Engineering Science), 2021, 55(5): 948-956.

URL:

http://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2021.05.015     OR     http://www.zjujournals.com/eng/Y2021/V55/I5/948


基于大边距度量学习的车辆再识别方法

在目前的车辆再识别方法中,车辆在拍摄过程中的多视角会导致特征嵌入空间中决策边界附近样本较难区分. 针对该问题,提出通过最大化边界距离提升车辆再识别准确率. 在特征表示阶段,设计了大边界损失度量函数,可以有效处理相似车辆的混淆问题;采用入侵叛逃采样策略,可以在训练样本中找出更容易混淆的难样本以更有针对性地训练网络,并加快网络的训练速度. 在车辆检索阶段,提出基于核函数的重排序方法,可以提高车辆再识别的准确率. 在3个公共数据集上的实验结果显示,车辆再识别的准确率得到提高,同时训练和推理效率得到改善. 理论分析和实验表明,大边距度量学习通过挖掘决策边界的难样本,可以有效解决车辆再识别中的多视角问题.


关键词: 车辆再识别,  特征嵌入,  度量学习,  损失函数,  采样策略,  核函数 
Fig.1 Overall network structure of large margin metric learning
Fig.2 Decision boundary in two dimensional feature space between different samples
Fig.3 Large margin metric learning example
Fig.4 Invader-defector sampling strategy
数据库 方法 Rank1 Rank5 mAP
Veri-776 Res 94.35 97.30 72.10
OS 94.61 97.50 72.50
OS+BN 95.40 97.90 74.50
OS+BN+LML 95.60 97.95 75.10
OS+BN+LML+IDS 96.45 98.68 80.55
OS+BN+LML+IDS+KER 96.81 98.95 80.95
VehicleID Res 85.45 94.93 77.22
OS 85.62 95.05 77.41
OS+BN 85.92 95.12 78.45
OS+BN+LML 85.95 95.15 80.58
OS+BN+LML+IDS 88.25 97.45 86.88
OS+BN+LML+IDS+KER 88.35 98.22 88.78
Tab.1 Ablation study on different datasets
Fig.5 Visualization of vehicle embedding features in three dimensional space
方法 mAP Rank1 Rank5
QD-DLF[11] 61.83 88.50 94.40
Part-Reg[10] 74.30 94.30 98.70
AAVER[9] 61.20 88.97 94.70
GSTE[6] 59.40 96.24 98.97
VANet[8] 66.34 89.78 95.99
LML(IDS) 80.55 96.45 98.68
LML(IDS)+KER 80.95 96.81 98.95
Tab.2 Accuracy comparison of proposed method with other methods on Veri-776 dataset
方法 Small Medium Large
mAP Rank1 mAP Rank1 mAP Rank1
QD-DLF[11] ? 74.69 ? 68.62 ? 63.54
Part-Reg[10] 76.54 72.32 74.63 70.66 68.41 64.14
AAVER[9] 61.50 78.40 ? 75.00 ? 74.20
GSTE[6] 75.40 75.90 74.30 74.80 72.40 74.00
VANet[8] ? 88.12 ? 83.17 ? 80.35
LML(IDS) 86.60 88.25 83.55 83.42 78.38 80.33
LML(IDS)+
KER
88.75 88.31 84.45 83.85 80.68 80.75
Tab.3 Accuracy comparison of proposed method with other methods on VehicleID dataset
方法 Small Medium Large
mAP Rank1 mAP Rank1 mAP Rank1
DRDL[21] 22.50 57.00 19.30 51.90 14.80 44.60
FDA-Net[18] 35.10 64.00 29.80 57.80 68.41 49.40
MLSL[22] 46.30 78.40 42.40 83.00 22.80 77.50
LML(IDS) 86.47 95.94 82.85 94.61 76.63 90.75
LML(IDS)+
KER
87.45 96.28 83.35 95.15 77.35 91.85
Tab.4 Accuracy comparison of proposed method with other methods on VERI-Wild dataset
Fig.6 Visualization of retrieval results
Fig.7 Failure case analysis of vehicle retrieval
方法 mAP p /M e v /(ms·image?1)
QD-DLF[11] 61.83 24 ? 11.19
Part-Reg[10] 74.30 48 130 ?
AAVER[9] 61.20 48 120 ?
GSTE[6] 59.40 24 ? ?
VANet[8] 66.34 24 200 ?
LML(Res,Hard) 80.35 24 150 1.67
LML(Res,IDS) 80.50 24 110 1.67
LML(OS,Hard) 80.46 2.7 150 1.12
LML(OS,IDS) 80.55 2.7 110 1.12
Tab.5 Time efficiency comparison of proposed method with other methods
[1]   LIU X C, LIU W, MA H D, et al. Large-scale vehicle re-identification in urban surveillance videos [C]// IEEE International Conference on Multimedia and Expo. Seattle: IEEE, 2016: 1-6.
[2]   LIU X C. A deep learning-based approach to progressive vehicle re-identification for urban surveillance [C]// 2016 European Conference on Computer Vision. Netherlands: Springer, 2016: 869-884.
[3]   ZHANG Y, LIU D, ZHA Z J. Improving triplet-wise training of convolutional neural network for vehicle re-identification [C]// 2017 IEEE International Conference on Multimedia and Expo. Hongkong: IEEE, 2017: 1386-1391.
[4]   ZHOU K Y, YANG Y X, CAVALLARO A, et al. Omni-scale feature learning for person re-identification [C]// 2019 International Conference of Computer Vision. Seoul: IEEE, 2019: 3701-3711.
[5]   HE K M, ZHANG X Y. Deep residual learning for image recognition [C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. Nevada: IEEE, 2016: 770-778.
[6]   BAI Y, LOU Y H, GAO F, et al Group-sensitive triplet embedding for vehicle re-identification[J]. IEEE Transactions on Multimedia, 2018, 2385- 2399
[7]   SIMONYAN K, ZISSERME N. Very deep convolutional networks for largescale image recognition [C]// 2015 International Conference on Learning Representations. San Diego: IEEE, 2015: 1-14.
[8]   CHU R H, SUN Y F. Vehicle re-identification with viewpoint-aware metric learning [C]// 2019 International Conference of Computer Vision. Seoul: IEEE, 2019: 8281-8290.
[9]   PIRAZ K. A dual-path model with adaptive attention for vehicle re-identification [C]// 2019 International Conference of Computer Vision. Seoul: IEEE, 2019: 6131-6140.
[10]   HE B. Part-regularized near-duplicate vehicle re-identification [C]// 2019 International Conference of Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 3992-4000.
[11]   ZHU J Q Vehicle re-identification using quadruple directional deep learning features[J]. IEEE Transactions on Intelligent Transportation System, 2019, 21 (1): 410- 420
[12]   GUO H, ZHAO C. Learning coarse-to-fine structured feature embedding for vehicle re-identification [C]// 2018 the 32nd AAAI Conference on Artificial Intelligence. New Orleans: AAAI, 2018: 6853-6860.
[13]   ZHENG Z, ZHENG L, YANG Y A discriminatively learned cnn embedding for personre-identification[J]. ACM Transactions on Multime-dia Computing, Communications, and Applications, 2017, 14 (1): 1- 20
[14]   LIU X, LIU W Provid: progressive and multimodal vehicle reidentification for large-scale urban surveillance[J]. IEEE Transactions on Multimedia, 2018, 20 (3): 645- 658
doi: 10.1109/TMM.2017.2751966
[15]   YU R, DOU Z, BAI S Hard-aware point-to-set deep metric for person re-identification[J]. Lecture Notes in Computer Science, 2018, 196- 212
[16]   CORTES C, VAPNIK V Support-vector networks[J]. Machine Learning, 1995, 20 (3): 273- 297
[17]   ZHONG Z, ZHENG L. Re-ranking person re-identification with k-reciprocal encoding [C]// 2017 International Conference of Computer Vision and Pattern Recognition. Honolulu: IEEE, 2019: 3652-3661.
[18]   LOU Y, BAI Y. Veri-wild: a large dataset and anew method for vehicle re-identification in the wild [C]// 2019 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2019: 3230-3238.
[19]   LIU H, TIAN Y. Deep relative distance learning: tell the difference between similar vehicles [C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2019: 2167–2175.
[20]   MAATEN L, HINTON G Visualizing data using t-SNE[J]. Journal of Machine Learning Review, 2008, 9: 2579- 2605
[21]   LIU Y, TIAN Y. Deep relative distance learning: tell the difference between similar vehicles [C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2019: 2167-2175.
[1] Fang LIU,Zhen WANG,Rui-di LIU,Kai WANG. Short-term forecasting method of wind power generation based on BP neural network with combined loss function[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(3): 594-600.
[2] QIN Hong-yuan, LIU Yi-ming, HUANG Dan. Peridynamic modelling and simulation for multiple crack propagation in brittle materials[J]. Journal of ZheJiang University (Engineering Science), 2018, 52(3): 497-503.
[3] JIA Song min, LU Ying bin, WANG Li jia, LI Xiu zhi, XU Tao. Mobile robot human tracking using hierarchical features[J]. Journal of ZheJiang University (Engineering Science), 2016, 50(9): 1677-1683.
[4] GU Hong, ZHAO Guang-zhou. Image retrieval and recognition based on generalized
local distance functions
[J]. Journal of ZheJiang University (Engineering Science), 2011, 45(4): 596-601.
[5] LI Xiao-run, WU Xiao-ming, ZHAO Liao-ying. Unsupervised nonlinear decomposing method of
hyperspectral imagery
[J]. Journal of ZheJiang University (Engineering Science), 2011, 45(4): 607-613.