Please wait a minute...
Journal of ZheJiang University (Engineering Science)  2022, Vol. 56 Issue (6): 1135-1143, 1167    DOI: 10.3785/j.issn.1008-973X.2022.06.010
    
Efficient convolution operators integrating motion information and tracking evaluation
Xun ZHANG1(),Jian-sheng LI1,*(),Wen OUYANG1,Run-ze CHEN1,Zhen JI1,Kai ZHENG2
1. Institute of Geographical Spatial Information, Information Engineering University, zhengzhou 450001, China
2. 73159 Troops, Quanzhou 362100, China
Download: HTML     PDF(2544KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

An efficient convolution operator integrating motion information and tracking evaluation was proposed, to solve the problems that an efficient convolution operators algorithm named by histogram of oriented gridients and color (ECO-HC) lacks the quality evaluation of tracking result and the supervision mechanism of filter model updating. Firstly, the Kalman filter was added to the ECO-HC positioning framework to carry out joint tracking of objects. A high-confidence discriminant indicator was designed to evaluate the tracking result of ECO-HC for each frame of video. If the current frame tracking results did not meet the confidence requirement, the weighted fusion value of the algorithm tracking results and the predicted value of Kalman filter would be used as the final target tracking results. Then, based on the filter template update strategy of the original algorithm, the quality evaluation information of the current frame tracking results was added.The filter template update was performed when the two conditions were true. Finally, the performance of the improved algorithm was evaluated on a open data set OTB-2015. The results showed that the overall tracking precision, success rate and tracking rate of the algorithm with proposed operator were superior to the original algorithm. The precision in the scene with motion blur, low resolution and out of view was increased by 3.0%, 3.5% and 2.8% respectively, and the success rate was increased by 3.8%, 2.1% and 4.0% respectively. The proposed algorithm not only ensured the real-time performance, but also improved the tracking performance in complex scenes.



Key wordsvisual object tracking      efficient convolution operator(ECO)      tracking evaluation      high-confidence discrimination indicator      Kalman filter     
Received: 14 July 2021      Published: 30 June 2022
CLC:  TP 391  
Corresponding Authors: Jian-sheng LI     E-mail: 1219233886@qq.com;ljs2021@vip.henu.edu.cn
Cite this article:

Xun ZHANG,Jian-sheng LI,Wen OUYANG,Run-ze CHEN,Zhen JI,Kai ZHENG. Efficient convolution operators integrating motion information and tracking evaluation. Journal of ZheJiang University (Engineering Science), 2022, 56(6): 1135-1143, 1167.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2022.06.010     OR     https://www.zjujournals.com/eng/Y2022/V56/I6/1135


融合运动信息和跟踪评价的高效卷积算子

针对基于方向梯度直方图与颜色命名的高效卷积算子(ECO-HC)算法缺少跟踪质量评价和滤波模板更新监督机制的问题,提出融合运动信息和跟踪评价的高效卷积算子. 将卡尔曼滤波器加入ECO-HC跟踪定位框架对目标执行联合跟踪,设计高置信度判别指标评价ECO-HC对每帧图像的跟踪效果,使用原始跟踪结果和卡尔曼滤波预测值的加权融合值,修正不满足判别指标的跟踪结果. 在滤波模板隔帧更新策略的基础上,加入当前帧跟踪结果质量评价信息,当2个条件同时满足时执行模板更新. 依托公开数据集OTB-2015评估算法性能,结果显示改进算法整体跟踪精确度、成功率和跟踪速率均优于原算法,在运动模糊、低分辨率、离开视野场景中的精确度分别提高3.0%、3.5%和2.8%,成功率分别提高3.8%、2.1%和4.0%. 改进算法在保证实时性的同时,有效提升了复杂场景下的跟踪效果.


关键词: 视觉目标跟踪,  高效卷积算子(ECO),  跟踪评价,  高置信度指标,  卡尔曼滤波器 
Fig.1 Tracking framework of improved ECO-HC algorithm
Fig.2 Tracking results of ECO-HC algorithm with Girl2 sequence
Fig.3 Response maps of ECO-HC algorithm with Girl2 sequence
$ {\beta }_{1},{\beta }_{2} $ P S $ {\beta }_{1},{\beta }_{2} $ P S
(0,0.45) 83.3 77.1 (0.6,0.45) 83.9 77.5
(0.55,0.45) 83.3 77.0 (0.6,0.50) 83.4 76.6
(0.6,0) 83.8 76.8 (0.65,0.45) 84.0 77.0
(0.6,0.40) 83.5 76.5 ? ? ?
Tab.1 Overall tracking results of control groups %
跟踪算法 P/% S/% R/(帧·s ?1
本研究 83.9 77.5 37.1
ECO-HC 83.3 76.1 36.5
DSST 67.9 60.0 60.5
SRDCF 79.0 72.8 3.4
SRDCFdecon 82.5 76.6 1.8
Tab.2 Overall tracking results and tracking rate of 5 tracking algorithms
Fig.4 Precision and success plots of one-pass evaluation for different algorithms on OTB2015 dataset
跟踪算法 P
IV OPR SV OCC DEF MB FM IPR OV BC LR
本研究 80.0 81.6 81.0 80.1 81.0 80.6 81.4 77.0 80.2 81.5 83.7
ECO-HC 78.7 80.3 79.6 78.2 79.8 77.6 80.2 76.4 77.4 81.7 80.2
DSST 71.5 64.4 63.3 58.9 53.3 56.7 55.2 69.1 48.1 70.4 56.7
SRDCF 78.6 74.2 74.1 73.0 72.8 76.7 76.9 74.5 59.7 77.5 65.5
SRDCFdecon 83.3 79.7 80.3 76.5 75.0 81.4 77.5 77.6 64.1 85.0 64.4
Tab.3 Tracking Precision results under different challenge sequences of 5 tracking algorithms %
跟踪算法 S
IV OPR SV OCC DEF MB FM IPR OV BC LR
本研究 76.5 73.5 73.0 75.0 73.6 78.6 77.0 68.1 73.3 77.5 72.1
ECO-HC 75.5 71.7 70.6 73.0 70.9 74.8 75.2 65.8 69.3 76.8 70.0
DSST 64.9 55.1 52.5 53.1 47.9 55.1 51.7 58.9 44.2 61.3 44.2
SRDCF 74.0 66.4 66.2 67.8 65.9 72.9 71.7 66.2 55.8 70.1 62.6
SRDCFdecon 78.9 72.0 73.3 72.5 67.3 79.9 73.2 69.8 64.1 78.6 61.9
Tab.4 Tracking success results under different challenge sequences of 5 tracking algorithms %
Fig.5 Tracking comparison of different algorithms in typical complex sequences
指标 EAO R f Acc
本研究 0.2805 21.1684 0.3967
ECO-HC 0.2693 23.8174 0.3828
Tab.5 Overall tracking results of tracking algorithms based on VOT2016
[1]   陈政. 智能监控系统中运动检测与目标跟踪模块的设计与实现[D]. 哈尔滨: 哈尔滨工业大学, 2020: 11-21.
CHEN Zheng. Design and implementation of motion detection and tracking module in intelligent surveilance system [D]. Harbin: Harbin Institute of Technology, 2020: 11-21.
[2]   黄石磊. 基于视觉计算的运动目标跟踪及异常行为分析[D]. 郑州: 郑州大学, 2014: 42-47.
HUANG Shi-lei. Moving target tracking and abnormal behavior analysis based on visual calculation [D]. Zhengzhou: Zhengzhou University, 2014: 42-47.
[3]   吴昊, 姜文斌, 陆增洁, 等 机器人视觉动态目标跟踪的最优方案研究[J]. 信息与电脑, 2020, (1): 129- 130
WU Hao, JIANG Wen-bin, LU Zeng-jie, et al Research on optimal scheme of robot vision dynamic target tracking[J]. China Computer and Communication, 2020, (1): 129- 130
[4]   程相博, 李建胜, 胡鹏, 等 改进的GOTURN算法[J]. 测绘科学技术学报, 2018, 35 (6): 605- 608
CHENG Xiang-bo, LI Jian-sheng, HU Peng, et al An improved algorithm of the generic object tracking using regression networks[J]. Journal of Geomatics Science and Technology, 2018, 35 (6): 605- 608
[5]   葛宝义, 左宪章, 胡永江 视觉目标跟踪方法研究综述[J]. 中国图象图形学报, 2018, 23 (8): 1091- 1107
GE Bao-yi, ZUO Xian-zhang, HU Yong-jiang Review of visual object tracking technology[J]. Journal of Image and Graphics, 2018, 23 (8): 1091- 1107
doi: 10.11834/jig.170604
[6]   BOLME D S, BEVERIDGE J R, DRAPER B A , et al. Visual object tracking using adaptive correlation filters [C]// 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Francisco: IEEE, 2010: 2544-2550.
[7]   HENRIQUES J F, CASEIRO R, MARTINS P, et al. Exploiting the circulant structure of tracking-by-detection with kernels[C]//Proceedings of European Conference on Computer Vision. Berlin: Springer Verlag, 2012: 702-715.
[8]   HENRIQUES J F, CASEIRO R, MARTINS P, et al High-speed tracking with kernelized correlation filters[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37 (3): 583- 596
doi: 10.1109/TPAMI.2014.2345390
[9]   DANELLJAN M, HÄGER G, KHAN F S, et al Discriminative scale space tracking[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39 (8): 1561- 1575
doi: 10.1109/TPAMI.2016.2609928
[10]   LI Y, ZHU J. A Scale adaptive kernel correlation filter tracker with feature integration [M]// AGAPITO L, BRONSTEIN M, ROTHER C, et al. Computer vision: ECCV 2014. [S.l.]: Springer, 2014: 254-265.
[11]   DANELLJAN M, HÄGER G, KHAN F S, et al. Learning spatially regularized correlation filters for visual tracking [C]// 2015 IEEE International Conference on Computer Vision. Santiago: IEEE, 2015: 4310-4318.
[12]   DANELLJAN M, HÄGER, KHAN F S, et al. Adaptive decontamination of the training set: a unified formulation for discriminative visual tracking [C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 1430-1438.
[13]   DANELLJAN M, HÄGER G, KHAN F S, et al. Beyond correlation filters: Learning continuous convolution operators for visual tracking [C]// European Conference on Computer Vision. Amsterdam: Springer, 2016: 472-488.
[14]   DANELLJAN M, BHAT G, KHAN F S, et al. ECO: efficient convolution operators for tracking [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 6638-6646.
[15]   FELZENSZWALB P F, GIRSHICK R B, MCALLESTER D, et al Object detection with discriminatively trained part-based models[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32 (9): 1627- 1645
doi: 10.1109/TPAMI.2009.167
[16]   DANELLJAN M, KHAN F S, FELSBERG M, et al. Adaptive color attributes for real-time visual tracking [C]// 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus: IEEE, 2014: 1090-1097.
[17]   WANG M, LIU Y, HUANG Z. Large margin object tracking with circulant feature maps [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 4800-4808.
[18]   周勇. 改进的卡尔曼滤波在目标跟踪中的应用[D]. 哈尔滨: 哈尔滨工程大学, 2016: 19-29.
ZHOU Yong. Application of the improved Kalman filter in target tracking [D]. Harbin: Harbin Institute of Technology, 2016: 19-29.
[19]   WU Y, LIM J, YANG M H Object tracking benchmark[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37 (9): 1834- 1848
doi: 10.1109/TPAMI.2014.2388226
[1] Si-peng WANG,Chang-ping DU,Guang-hua SONG,Yao ZHENG. Indoor positioning method of UAV based on improved MSCKF algorithm[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(4): 711-717.
[2] Cheng-qian GUO,Gang MA,Jiang-zhou MEI,Gui-ke ZHANG,Hong-bi LI,Wei ZHOU. Exterior deformation reconstruction of rockfill dam based on InSAR and multi-source data fusion[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(2): 347-355.
[3] Chuan-hui WU,Jia LIAO,Shi-yong XIONG,Ying-jie NIU,Bo ZHOU. Contour matching method of groove track based on laser sensor[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(9): 1607-1614.
[4] Ke-bing LEI,Zi-qiang CHEN. Estimation of state of charge of battery based on improved multi-innovation extended Kalman filter[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(10): 1978-1985.
[5] Kang-hao WANG,Hai-bing YIN,Xiao-feng HUANG. Visual object tracking based on policy gradient[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(10): 1923-1928.
[6] WANG Ri-jun, ZHAO Chang-jun, BAI Yue, ZENG Zhi-qiang, DU Wen-hua, DUAN Neng-quan. Faults detection and self-reconfiguration for execution units of Hex-Rotor unmanned aerial vehicle based on multiple fault classification[J]. Journal of ZheJiang University (Engineering Science), 2018, 52(7): 1406-1414.
[7] LIU Jing-ming, HUANG Ping-jie, HOU Di-bo, ZHANG Guang-xin, ZHANG Hong-jian. Dynamic pollutant concentration correction method for river sudden pollution[J]. Journal of ZheJiang University (Engineering Science), 2017, 51(12): 2459-2465.
[8] CUI Ren-jie, MENG Tao, HUO Jun-hai, JIN Zhong-he. On-orbit calibration technique for residual magnetism fluctuation of micro-satellite[J]. Journal of ZheJiang University (Engineering Science), 2017, 51(12): 2444-2450.
[9] YE Xiao-wei, LIU Tan, DONG Chuan-zhi, CHEN Bin. Structural damage detection based on Kalman filter and neutral axis location[J]. Journal of ZheJiang University (Engineering Science), 2017, 51(10): 2012-2018.
[10] SONG Kai chen, ZENG Yao, YE Ling yun . Vortex signal processing method based on multi-sensor information fusion[J]. Journal of ZheJiang University (Engineering Science), 2016, 50(7): 1307-1312.
[11] MO Yuan fu, YU De xin, SONG Jun, GUO Ya juan. Beacon message generating strategy based on channel load preset threshold in VANET environment[J]. Journal of ZheJiang University (Engineering Science), 2016, 50(1): 21-26.
[12] ZHU Guang-ming, JIANG Rong-xin, ZHOU Fan, TIAN Xiang, CHEN Yao-wu. Robust Kalman filtering algorithm with estimation of measurement biases[J]. Journal of ZheJiang University (Engineering Science), 2015, 49(7): 1343-1349.
[13] FU Xing-wei, WU Gong-ping, ZHOU Peng, YU Na. Energy-consumption estimation of inspection robot based on Kalman filter[J]. Journal of ZheJiang University (Engineering Science), 2015, 49(4): 670-675.
[14] LU Guo-sheng, LI Li-yan, ZHAO Min-jian. Design and analysis of global navigation satellite system vector carrier loop[J]. Journal of ZheJiang University (Engineering Science), 2015, 49(1): 20-26.
[15] ZHENG Chi, XIANG Zhi-yu, LIU Ji-lin. Monocular vision odometry based on the fusion of optical flow and feature points matching[J]. Journal of ZheJiang University (Engineering Science), 2014, 48(2): 279-284.