Please wait a minute...
浙江大学学报(工学版)
Journal of ZheJiang University (Engineering Science)  2022, Vol. 56 Issue (9): 1824-1832    DOI: 10.3785/j.issn.1008-973X.2022.09.016
    
UAV object tracking algorithm based on response and filter deviation-aware regularization
Hai-jun WANG(),Sheng-yan ZHANG,Yu-jie DU
Key Laboratory of Aviation Information and Control in University of Shandong, Binzhou University, Binzhou 256603, China
Download: HTML     PDF(805KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

A real-time unmanned aerial vehicle (UAV) object tracking algorithm based on the response and filter deviation-aware regularization was proposed, aiming at the problem that targets were easily subject to the huge variation of appearance and various change of viewpoint interference in the UAV sequences. According to the consistency of response and correlation filter difference between video frames, the variation of correlation response and filter difference were modeled. Furthermore, an objective function with constraint scheme was constructed, which can learn variation of object appearance and filter. Meanwhile, an auxiliary variable based on the response and filter deviation-aware regularization was introduced to build an optimization function and alternating direction method of multipliers (ADMM) was used to optimize the solution of the correlation filter and auxiliary variable. To validate the effectiveness of the proposed algorithm, comparison experiments with other 9 algorithms were performed on three UAV tracking benchmarks, including DTB70、UAV123@10 fps and UAVDT, in terms of precision and success rate. Experimental results show that the proposed algorithm has good robustness for occlusion, deformation and view variation and can effectively track the target with an average speed of 39.0 frames of second.

Key words： unmanned aerial vehicle (UAV)      correlation filter      visual object tracking      response deviation-aware regularization      filter deviation     
Received: 06 August 2021      Published: 28 September 2022
CLC:  TP 391  
Fund:  山东省自然科学基金资助项目（ZR2020MF142, ZR2019PF021）；滨州学院博士启动基金资助项目（2021Y04）；滨州学院重大科研基金资助项目（2019ZD03）；滨州学院社会服务基金资助项目（BZXYSFW201805）
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Hai-jun WANG
Sheng-yan ZHANG
Yu-jie DU
Cite this article:

Hai-jun WANG,Sheng-yan ZHANG,Yu-jie DU. UAV object tracking algorithm based on response and filter deviation-aware regularization. Journal of ZheJiang University (Engineering Science), 2022, 56(9): 1824-1832.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2022.09.016     OR     https://www.zjujournals.com/eng/Y2022/V56/I9/1824


响应和滤波器偏差感知约束的无人机目标跟踪算法

针对无人机视觉跟踪任务中目标外观变化大、视野角度多变问题，提出基于响应和滤波器偏差感知约束的无人机实时目标跟踪算法. 该算法根据视频帧间响应差和滤波器变化的一致性，通过建模前后帧响应差和滤波器的变化，建立基于响应偏差感知和帧间滤波器偏差约束机制的目标函数，学习目标的外观变化和滤波器的帧间变化. 引入辅助变量构建优化函数，采用交替方向乘子法（ADMM）将计算目标问题转化为求相关滤波器和辅助变量的最优解. 采用跟踪准确度和成功率指标，将所提算法与其他9种算法在DTB70、UAV123@10 fps和UAVDT等3个无人机视频数据库上进行对比实验. 实验结果表明，所提算法对遮挡、形变、角度变化等干扰属性均具有良好的鲁棒性，跟踪平均速度达到39.0帧/s，能够有效跟踪无人机目标.


关键词： 无人机 (UAV),  相关滤波,  视觉目标跟踪,  响应偏差感知约束,  滤波器偏差 
算法 P Rs
DTB70 UAV123@10 fps UAVDT DTB70 UAV123@10 fps UAVDT
RDAR 0.671 0.672 0.724 0.462 0.488 0.458
ECO-HC 0.643 0.634 0.681 0.453 0.462 0.410
MCCT_H 0.604 0.596 0.667 0.405 0.433 0.402
BACF 0.590 0.572 0.686 0.402 0.413 0.433
fDSST 0.534 0.516 0.666 0.357 0.379 0.383
SAMF_CA 0.532 0.523 0.564 0.346 0.365 0.304
SAMF 0.519 0.466 0.579 0.340 0.326 0.312
SRDCF 0.512 0.575 0.658 0.363 0.423 0.419
SRDCFdecon 0.504 0.584 0.643 0.351 0.429 0.410
Staple 0.365 0.456 0.665 0.265 0.342 0.383
Tab.1 Comparison in terms of precision and success rates for ten tracking algorithms on three UAV datasets
算法 P
ARV BC DEF FCM IPR MB OCC OPR OV SV SOA
RDAR 0.600 0.589 0.656 0.705 0.613 0.675 0.582 0.434 0.628 0.661 0.700
BACF 0.392 0.545 0.448 0.636 0.547 0.639 0.515 0.266 0.650 0.533 0.624
MCCT_H 0.495 0.484 0.550 0.621 0.552 0.502 0.571 0.383 0.573 0.643 0.606
ECO-HC 0.506 0.567 0.584 0.680 0.568 0.640 0.641 0.430 0.557 0.530 0.667
SRDCFdecon 0.343 0.449 0.283 0.574 0.430 0.500 0.456 0.193 0.570 0.473 0.564
Tab.2 Comparison in terms of precision of five tracking algorithms for different attributes on DTB70 dataset
算法 P
ARC BC CM FM FOC IV LR OV POC SV SOB VC
RDAR 0.587 0.469 0.635 0.531 0.464 0.568 0.568 0.547 0.613 0.630 0.678 0.591
BACF 0.478 0.425 0.532 0.407 0.336 0.430 0.431 0.421 0.467 0.525 0.605 0.491
MCCT_H 0.493 0.469 0.544 0.361 0.421 0.477 0.455 0.493 0.542 0.547 0.627 0.484
ECO-HC 0.558 0.511 0.609 0.487 0.454 0.507 0.527 0.522 0.556 0.587 0.637 0.548
SRDCFdecon 0.476 0.427 0.536 0.403 0.427 0.423 0.436 0.483 0.514 0.535 0.621 0.478
Tab.3 Comparison in terms of precision of five tracking algorithms for different attributes on UAV123@10 pfs dataset
算法 P
BC CM IV LO LTT OB OM SV SO
RDAR 0.632 0.687 0.761 0.549 0.877 0.720 0.641 0.647 0.808
BACF 0.599 0.614 0.739 0.488 0.886 0.699 0.604 0.604 0.770
MCCT_H 0.571 0.622 0.703 0.482 0.925 0.667 0.561 0.594 0.796
ECO-HC 0.607 0.647 0.723 0.504 0.924 0.669 0.596 0.607 0.767
SRDCFdecon 0.533 0.588 0.690 0.433 0.812 0.650 0.560 0.565 0.716
Tab.4 Comparison in terms of precision of five tracking algorithms for different attributes on UAVDT dataset
算法 Rs
ARV BC DEF FCM IPR MB OCC OPR OV SV SOA
RDAR 0.396 0.379 0.434 0.483 0.420 0.458 0.413 0.318 0.407 0.465 0.467
BACF 0.273 0.337 0.302 0.436 0.371 0.412 0.348 0.203 0.419 0.392 0.411
MCCT_H 0.334 0.296 0.354 0.410 0.376 0.334 0.377 0.243 0.349 0.439 0.399
ECO-HC 0.376 0.349 0.404 0.469 0.410 0.434 0.431 0.319 0.416 0.430 0.446
SRDCFdecon 0.250 0.285 0.196 0.398 0.311 0.331 0.308 0.150 0.384 0.351 0.369
Tab.5 Comparison in terms of success rate of five tracking algorithms for different attributes on DTB70 dataset
算法 Rs
ARC BC CM FM FOC IV LR OV POC SV SOB VC
RDAR 0.412 0.302 0.464 0.355 0.257 0.391 0.342 0.404 0.427 0.453 0.478 0.422
BACF 0.334 0.275 0.397 0.275 0.173 0.310 0.248 0.321 0.327 0.374 0.424 0.353
MCCT_H 0.357 0.305 0.407 0.260 0.236 0.342 0.257 0.365 0.376 0.396 0.451 0.361
ECO-HC 0.392 0.339 0.449 0.332 0.247 0.362 0.299 0.387 0.391 0.424 0.464 0.400
SRDCFdecon 0.344 0.293 0.399 0.276 0.231 0.314 0.247 0.356 0.361 0.390 0.440 0.360
Tab.6 Comparison in terms of success rate of five tracking algorithms for different attributes on UAV123@10 pfs dataset
Fig.1 Comparison of representative tracking results by different algorithms on DTB70、UAV123@10 fps、UAVDT
算法 Rs
BC CM IV LO LTT OB OM SV SO
RDAR 0.399 0.435 0.465 0.386 0.577 0.442 0.397 0.435 0.464
BACF 0.367 0.387 0.460 0.340 0.582 0.443 0.371 0.408 0.428
MCCT_H 0.343 0.367 0.415 0.348 0.565 0.390 0.343 0.384 0.389
ECO-HC 0.364 0.379 0.434 0.348 0.573 0.391 0.358 0.390 0.375
SRDCFdecon 0.339 0.374 0.430 0.322 0.515 0.395 0.351 0.389 0.410
Tab.7 Comparison in terms of success rate of five tracking algorithms for different attributes on UAVDT dataset
算法 v/(帧·s?1) 算法 v/(帧·s?1)
RDAR 39.0 SAMF_CA 9.1
ECO-HC 62.2 SAMF 10.0
MCCT_H 59.0 SRDCF 10.7
BACF 46.5 SRDCFdecon 6.0
fDSST 132.0 Staple 62.5
Tab.8 Comparison of tracking speed for different tracking algorithms on DTB70 dataset
Fig.2 Comparison of tracking performance on UAVDT with different parameters
模块 P Rs
DTB70 UAV123@10 fps UAVDT DTB70 UAV123@10 fps UAVDT
BACF 0.590 0.572 0.686 0.402 0.413 0.433
BACF+RD 0.648 0.663 0.713 0.451 0.484 0.457
BACF+FD 0.667 0.649 0.671 0.458 0.470 0.453
RDAR(BACF+RD+CF) 0.671 0.672 0.724 0.462 0.488 0.458
Tab.9 Comparison of tracking performance with different modules on three UAV datasets
[1]   刘芳, 杨安喆, 吴志威 基于自适应Siamese网络的无人机目标跟踪算法[J]. 航空学报, 2020, 41 (1): 323423
LIU Fang, YANG An-zhe, WU Zhi-wei Adaptive Siamese network based UAV target tracking algorithm[J]. Acta Aeronautica et Astronautica Sinica, 2020, 41 (1): 323423
[2]   孙锐, 方林凤, 梁启丽, 等 孪生网络框架下融合显著性和干扰在线学习的航拍目标跟踪算法[J]. 电子与信息学报, 2021, 43 (5): 1414- 1423
SUN Rui, FANG Lin-feng, LIANG Qi-li, et al Siamese network combined learning saliency and online leaning interference for aerial object tracking algorithm[J]. Journal of Electronics and Information Technology, 2021, 43 (5): 1414- 1423
doi: 10.11999/JEIT200140
[3]   赵燕伟, 张健, 周仙明, 等 基于视觉−磁引导的无人机动态跟踪与精准着陆[J]. 浙江大学学报:工学版, 2021, 55 (1): 96- 108
ZHAO Yan-wei, ZHANG Jian, ZHOU Xian-ming, et al Dynamic tracking and precise landing of UAV based on visual magnetic guidance[J]. Journal of Zhejiang University: Engineering Science, 2021, 55 (1): 96- 108
[4]   刘芳, 王洪娟, 黄光伟, 等 基于自适应深度网络的无人机目标跟踪算法[J]. 航空学报, 2019, 40 (3): 322332
LIU Fang, WANG Hong-juan, HUANG Guang-wei, et al UAV target tracking algorithm based on adaptive depth network[J]. Acta Aeronautica et Astronautica Sinica, 2019, 40 (3): 322332
[5]   FU C H, HE Y J, LIN F L, et al Robust multi-kernelized correlators for UAV tracking with adaptive context analysis and dynamic weighted filters[J]. Neural Computing and Applications, 2020, 32: 12591- 12607
doi: 10.1007/s00521-020-04716-x
[6]   LI Y M, FU C H, HUANG Z Y, et al Intermittent contextual learning for keyfilter-aware UAV object tracking using deep convolutional feature[J]. IEEE Transactions on Multimedia, 2021, 23: 810- 822
doi: 10.1109/TMM.2020.2990064
[7]   HE Y J, FU C H, LIN F L, et al. Towards robust visual tracking for unmanned aerial vehicle with tri-attentional correlation filters [C]// 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Las Vegas: IEEE, 2020: 1575-1582.
[8]   BOLME D S, BEVERIDGE J R, DRAPER B A, et al. Visual object tracking using adaptive correlation filters [C]// 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Francisco: IEEE, 2010: 2544-2550.
[9]   HENRIQUES J F, CASEIRO R, MARTINS P, et al High-speed tracking with kernelized correlation filters[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37 (3): 583- 596
doi: 10.1109/TPAMI.2014.2345390
[10]   GALOOGAHI H K, FAGG A, LUCEY S. Learning background-aware correlation filters for visual tracking [C]// 2017 IEEE International Conference on Computer Vision (ICCV). Venice: IEEE, 2017: 1144-1152.
[11]   LI F, TIAN C, ZUO W M, et al. Learning spatial-temporal regularized correlation filters for visual tracking [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (ICCV). Salt Lake City: IEEE, 2018: 4904-4913.
[12]   DAI K, WANG D, LU H C, et al. Visual tracking via adaptive spatially-regularized correlation filters [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach: IEEE, 2019: 4665-4674.
[13]   MA C, HUANG J B, YANG X K, et al Robust visual tracking via hierarchical convolutional features[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41 (11): 2709- 2723
doi: 10.1109/TPAMI.2018.2865311
[14]   FU C H, YE J J, XU J T, et al Disruptor-aware interval-based response inconsistency for correlation filters in real-time aerial tracking[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59 (8): 6301- 6313
doi: 10.1109/TGRS.2020.3030265
[15]   DANELLJAN M, BHAT G, KHAN F S, et al. ECO: efficient convolution operators for tracking [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu: IEEE, 2017: 6931-6939.
[16]   WANG N, ZHOU W G, TIAN Q, et al. Multi-cue correlation filters for robust visual tracking [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 4844-4853.
[17]   DANELLJAN M, HÄGER G, KHAN F S, et al. Discriminative scale space tracking [J] IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(8): 1561-1575.
[18]   LI Y, ZHU J. A scale adaptive kernel correlation filter tracker with feature integration [C]// Computer Vision: ECCV 2014 Workshops. [S. l.]: Springer, 2015: 254-265.
[19]   MUELLER M, SMITH N, GHANEM B. Context-aware correlation filter tracking [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu: IEEE, 2017: 1387-1395.
[20]   DANELLJAN M, HÄGER G, KHAN F S, et al. Learning spatially regularized correlation filters for visual tracking [C]// 2015 IEEE International Conference on Computer Vision (ICCV). Santiago: IEEE, 2015: 4310-4318.
[21]   DANELLJAN M, HÄGER G, KHAN F S, et al. Adaptive decontamination of the training set: a unified formulation for discriminative visual tracking [C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE, 2016: 1430-1438.
[1] Xun ZHANG,Jian-sheng LI,Wen OUYANG,Run-ze CHEN,Zhen JI,Kai ZHENG. Efficient convolution operators integrating motion information and tracking evaluation[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(6): 1135-1143, 1167.
[2] Si-peng WANG,Chang-ping DU,Guang-hua SONG,Yao ZHENG. Indoor positioning method of UAV based on improved MSCKF algorithm[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(4): 711-717.
[3] Si-xiao WANG,Wen-jun ZHAO,Hao ZHANG,Yong GAO,Pu-sen LI. Cascade TD-PID control algorithm for coaxial anti-propeller unmanned aerial vehicle based on tracking differentiator[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(12): 2359-2364.
[4] Yun-xu BAI,Xin-jiang LU,Rui LUO. Adaptive context-aware target tracking method[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(10): 1834-1846.
[5] Chang-zhen XIONG,Yan LU,Jia-qing YAN. Visual tracking algorithm based on anisotropic Gaussian distribution[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(2): 301-310.
[6] Kang-hao WANG,Hai-bing YIN,Xiao-feng HUANG. Visual object tracking based on policy gradient[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(10): 1923-1928.
[7] Chang-zhen XIONG,Run-ling WANG,Jian-cheng ZOU. Real-time tracking algorithm based on multiple Gaussian-distribution correlation filters[J]. Journal of ZheJiang University (Engineering Science), 2019, 53(8): 1488-1495.
[8] Zhi-peng XI,Zhuo LOU,Xiao-xia LI,Yan SUN,Qiang YANG,Wen-jun YAN. Vision-based localization and navigation for UAV inspection in photovoltaic farms[J]. Journal of ZheJiang University (Engineering Science), 2019, 53(5): 880-888.
[9] TIAN Hua, ZHAO Wen jie, FANG Zhou, LI Ping. Guidance strategy of unpowered landing based on energy management for unmanned aerial vehicle[J]. Journal of ZheJiang University (Engineering Science), 2015, 49(10): 1999-2004.