Please wait a minute...
Journal of ZheJiang University (Engineering Science)  2021, Vol. 55 Issue (6): 1056-1064    DOI: 10.3785/j.issn.1008.973X.2021.06.005
Multi-target tracking of vehicles based on optimized DeepSort
Li-sheng JIN1,2(),Qiang HUA3,Bai-cang GUO1,Xian-yi XIE1,*(),Fu-gang YAN3,Bo-tao WU4
1. School of Vehicle and Energy, Yanshan University, Qinhuangdao 066004, China
2. Hebei Key Laboratory of Special Delivery Equipment, Yanshan University, Qinhuangdao 066004, China
3. Transportation College, Jilin University, Changchun 130022, China
4. Department of Automotive Engineering, Hebei Institute of Mechanical and Electrical Technology, Xingtai 054000, China
Download: HTML     PDF(1014KB) HTML
Export: BibTeX | EndNote (RIS)      


A front multi-vehicle target tracking algorithm optimized by DeepSort was proposed in order to improve the awareness of autonomous vehicles to the surrounding environment. Gaussian YOLO v3 model was adopted as the front-end target detector, and training was based on DarkNet-53 backbone network. Gaussian YOLO v3-Vehicle, a detector specially designed for vehicles was obtained, which improved the vehicle detection accuracy by 3%. The augmented VeRi data set was proposed to conduct the re-recognition pre-training in order to overcome the shortcomings that the traditional pre-training model doesn't target vehicles. A new loss function combining the central loss function and the cross entropy loss function was proposed, which can make the target features extracted by the network become better in-class aggregation and inter-class resolution. Actual road videos in different environments were collected in the test part, and CLEAR MOT evaluation index was used for performance evaluation. Results showed a 1% increase in tracking accuracy and a 4% reduction in identity switching times compared with the benchmark DeepSort YOLO v3.

Key wordsautonomous vehicle      environment perception      deep learning      optimized DeepSort algorithm      object tracking     
Received: 24 July 2020      Published: 30 July 2021
CLC:  U 462  
Fund:  国家重点研发计划资助项目(2018YFB1600501);国家自然科学基金资助项目(52072333);国家自然科学基金区域创新发展联合基金资助项目(U19A2069);河北省省级科技计划资助项目(20310801D,E2020203092,F2021203107)
Corresponding Authors: Xian-yi XIE     E-mail:;
Cite this article:

Li-sheng JIN,Qiang HUA,Bai-cang GUO,Xian-yi XIE,Fu-gang YAN,Bo-tao WU. Multi-target tracking of vehicles based on optimized DeepSort. Journal of ZheJiang University (Engineering Science), 2021, 55(6): 1056-1064.

URL:     OR


为了提升自动驾驶汽车对周边环境的感知能力,提出优化DeepSort的前方多车辆目标跟踪算法. 采用Gaussian YOLO v3作为前端目标检测器,基于DarkNet-53骨干网络训练,获得专门针对车辆的检测器Gaussian YOLO v3-vehicle,使车辆检测准确率提升3%. 为了克服传统预训练模型没有针对车辆类别的缺点,提出采用扩增后的VeRi数据集进行重识别预训练. 提出结合中心损失函数与交叉熵损失函数的新损失函数,使网络提取的目标特征有更好的类内聚合以及类间分辨能力. 试验部分采集不同环境的实际道路视频,采用CLEAR MOT评价指标进行性能评估. 结果表明,与基准DeepSort YOLO v3相比,跟踪准确度提升1%,身份切换次数减少4%.

关键词: 自动驾驶,  环境感知,  深度学习,  优化DeepSort算法,  目标跟踪 
Fig.1 Algorithm flow chart of DeepSort
层名称 权重尺寸 输出尺寸
Conv1 3×3/1 32×128×64
Conv2 3×3/1 32×128×64
Max Pool 3 3×3/2 32×64×32
Residual 4 3×3/1 32×64×32
Residual 5 3×3/1 32×64×32
Residual 6 3×3/2 64×32×16
Residual 7 3×3/1 64×32×16
Residual 8 3×3/2 128×16×8
Residual 9 3×3/1 128×16×8
Dense 10 128
BN 128
Tab.1 Re-identify network structure
Fig.2 Comparison of image data quality before and after enhancement
Fig.3 YOLO v3 network structure
Fig.4 Variation diagram of classification accuracy of vehicle re-identification training
Fig.5 Gaussian YOLO v3 loss curve
Fig.6 Visualization of shadow traffic scene environment
Fig.7 Visualization of continuous change of vehicle shape
Fig.8 Visualization of crowded environments in traffic scene
Unsup Track[4] 61.7 78.3
Lif-T[4] 60.5 79.0
ISE-MOT17R[4] 60.1 78.2
msot[5] 59.2 78.0
EAMTT[27] 52.5 78.8
POI[27] 66.1 77.1
SORT[27] 59.8 79.6
基准DeepSort[27] 61.4 79.1
本文算法 62.5 78.9
Tab.2 Comparison of MOTA and MOTP for multi-objective tracking algorithms
[1]   王世峰, 戴祥, 徐宁 无人驾驶汽车环境感知技术综述[J]. 长春理工大学学报: 自然科学版, 2017, 40 (1): 1- 6
WANG Shi-feng, DAI Xiang, XU Ning Overview of driverless car environment perception technology[J]. Journal of Changchun University of Science and Technology: Natural Science Edition, 2017, 40 (1): 1- 6
[2]   李玺, 查宇飞, 张天柱 深度学习的目标跟踪算法综述[J]. 中国图象图形学报, 2019, 24 (12): 2057- 2080
LI Xi, ZHA Yu-fei, ZHANG Tian-zhu Overview of deep learning target tracking algorithms[J]. Chinese Journal of Image and Graphics, 2019, 24 (12): 2057- 2080
doi: 10.11834/jig.190372
[3]   储琪. 基于深度学习的视频多目标跟踪算法研究[D]. 合肥: 中国科学技术大学, 2019.
CHU Qi. Research on video multi-target tracking algorithm based on deep learning[D]. Hefei: University of Science and Technology of China, 2019.
[4]   KIM C, LI F, CIPTADI A, et al. Multiple hypothesis tracking revisited[C]// Proceedings of the IEEE International Conference on Computer Vision. Santiago: IEEE, 2015: 4696–4704.
[5]   HU H, ZHOU L, GUAN Q, et al An automatic tracking method for multiple cells based on multi-feature fusion[J]. IEEE Access, 2018, 6: 69782- 69793
doi: 10.1109/ACCESS.2018.2880563
[6]   LEAL-TAIXÉ L, FERRER C C, SCHINDLER K. Learning by tracking: siamese CNN for robust target association[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Las Vegas: IEEE, 2016: 33–40.
[7]   ZHOU H, OUYANG W, CHENG J, et al Deep continuous conditional random fields with asymmetric inter-object constraints for online multi-object tracking[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2018, 29 (4): 1011- 1022
[8]   ULLAH M, MOHAMMED A K, CHEIKH F A, et al. A hierarchical feature model for multi-target tracking[C]// Proceedings of the 2017 IEEE International Conference on Image Processing. Beijing: IEEE, 2017: 2612–2616.
[9]   SHARMA S, ANSARI J A, MURTHY J K, et al. Beyond pixels: Leveraging geometry and shape cues for online multi-object tracking[C]// Proceedings of the 2018 IEEE International Conference on Mechatronics, Robotics and Automation. Brisbane: IEEE, 2018: 3508–3515.
[10]   CHU Q, OUYANG W, LI H, et al. Online multi-object tracking using CNN-based single object tracker with spatial-temporal attention mechanism[C]// 2017 IEEE International Conference on Computer Vision. Venie: IEEE, 2017: 4836–4845.
[11]   MILAN A, REZATOFIGHI S H, DICK A, et al. Online multi-target tracking using recurrent neural networks[C]// National Conference on Artificial Intelligence. San Francisco: AAAI Press, 2017: 4225-4232.
[12]   MA C, YANG C, YANG F, et al. trajectory factory: tracklet cleaving and re-connection by deep siamese bi-GRU for multiple object tracking[C]// 2018 IEEE International Conference on Multimedia and Expo. San Diego: IEEE, 2018: 1-6.
[13]   ZHU J, YANG H, LIU N, et al. Online multi-object tracking with dual matching attention networks[C]// 2018 European Conference on Computer Vision. Munich: [s. n.], 2018: 366–382.
[14]   WOJKE N, BEWLEY A, PAULUS D. Simple online and real time tracking with a deep association metric[C]// 2017 IEEE International Conference on Image Processing. [S. l.]: IEEE, 2017: 3645-3649.
[15]   解耘宇. 基于扩展卡尔曼滤波的单目视觉轨迹跟踪方法的研究[D]. 北京: 华北电力大学, 2017.
XIE Yun-yu. Research on monocular vision trajectory tracking method based on extended Kalman filter[D]. Beijing: North China Electric Power University, 2017.
[16]   BISHOP C M. Pattern recognition and machine learning (information science and statistics)[M]. New York: Springer, 2006.
[17]   WOJKE N, BEWLEY A. Deep cosine metric learning for person re-identification[C]// IEEE Winter Conference on Applications of Computer Vision. Lake Tahoe: IEEE, 2018: 748-756.
[18]   KRIZHEVSKY A, SUTSKEVER I, HINTON G E Imagenet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60 (6): 84- 90
doi: 10.1145/3065386
[19]   ZAGORUYKO S, KOMODAKIS N. Wide residual networks [C]// 2016 Proceedings of the British Machine Vision Conference. York: DBLP, 2016: 1-15.
[20]   WEN Y, ZHANG K, LI Z, et al. A discriminative feature learning approach for deep face recognition[C]// European Conference on Computer Vision. Amsterdam: Springer, 2016: 499-515.
[21]   CHARU C A. Neural networks and deep learning[M]. Cham: Springer, 2018: 48-51.
[22]   刘鑫辰. 城市视频监控网络中车辆搜索关键技术研究[D]. 北京: 北京邮电大学, 2018.
LIU Xin-chen. Research on key technologies of vehicle search in urban video surveillance network[D]. Beijing: Beijing University of Posts and Telecommunications, 2018.
[23]   REN S, HE K, GIRSHICK R, et al Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39 (6): 1137- 1149
doi: 10.1109/TPAMI.2016.2577031
[24]   LIN T Y, DOLLAR P, GIRSHICK R. Feature pyramid networks for object detection[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 936-944.
[25]   REDMON J, FARHADI A. YOLO v3: an incremental improvement [EB/OL]. [2020-05-31].
[26]   CHOI J, CHUN D, KIM H. Gaussian YOLO v3: an accurate and fast object detector using localization uncertainty for autonomous driving[C]// International Conference on Computer Vision. Seoul: IEEE, 2019: 502-511.
[1] Li HE,Shan-min PANG. Face reconstruction from voice based on age-supervised learning and face prior information[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(5): 1006-1016.
[2] Xue-qin ZHANG,Tian-ren LI. Breast cancer pathological image classification based on Cycle-GAN and improved DPN network[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(4): 727-735.
[3] Jing-hui CHU,Li-dong SHI,Pei-guang JING,Wei LV. Context-aware knowledge distillation network for object detection[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(3): 503-509.
[4] Ruo-ran CHENG,Xiao-li ZHAO,Hao-jun ZHOU,Han-chen YE. Review of Chinese font style transfer research based on deep learning[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(3): 510-519, 530.
[5] Tong CHEN,Jian-feng GUO,Xin-zhong HAN,Xue-li XIE,Jian-xiang XI. Visible and infrared image matching method based on generative adversarial model[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(1): 63-74.
[6] Song REN,Qian-wen ZHU,Xin-yue TU,Chao DENG,Xiao-shu WANG. Lining disease identification of highway tunnel based on deep learning[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(1): 92-99.
[7] Ning-bo CAO,Li-ying ZHAO. Decision-making method of autonomous vehicles for right of way on road segments[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(1): 118-127.
[8] Xing LIU,Jian-bo YU. Attention convolutional GRU-based autoencoder and its application in industrial process monitoring[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(9): 1643-1651.
[9] Xue-yun CHEN,Xiao-qiao HUANG,Li XIE. Classification and detection method of blood cells images based on multi-scale conditional generative adversarial network[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(9): 1772-1781.
[10] Jia-cheng LIU,Jun-zhong JI. Classification method of fMRI data based on broad learning system[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(7): 1270-1278.
[11] Guo-min QIAN,Jun-sheng FAN,Chun-guang HE,Li-hui ZHANG,Dian-hai WANG. Optimization of spatial-temporal resources at intersections under environment of mixed traffic flow with connected and autonomous vehicles and human-driven vehicles[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(6): 1019-1026.
[12] Jia-hui XU,Jing-chang WANG,Ling CHEN,Yong WU. Surface water quality prediction model based on graph neural network[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(4): 601-607.
[13] Hong-li WANG,Bin GUO,Si-cong LIU,Jia-qi LIU,Yun-gang WU,Zhi-wen YU. End context-adaptative deep sensing model with edge-end collaboration[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(4): 626-638.
[14] Teng ZHANG,Xin-long JIANG,Yi-qiang CHEN,Qian CHEN,Tao-mian MI,Piu CHAN. Wrist attitude-based Parkinson's disease ON/OFF state assessment after medication[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(4): 639-647.
[15] Li-feng XU,Hai-fan HUANG,Wei-long DING,Yu-lei FAN. Detection of small fruit target based on improved DenseNet[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(2): 377-385.