UAV target tracking algorithm based on event camera

doi:10.3785/j.issn.1008-9497.2022.01.002

Journal of Zhejiang University (Science Edition)

2022, Vol. 49

Issue (1): 10-18 DOI: 10.3785/j.issn.1008-9497.2022.01.002

Graphics Simulation and Target Tracking

UAV target tracking algorithm based on event camera

Qiang ZHU,Chaoyi WANG,Jiqing ZHANG,Baocai YIN,Xiaopeng WEI(

),Xin YANG(

)

School of Computer Science and Technology，Dalian University of Technology，Dalian 116000，China

Download:

HTML( 7 )

PDF(3961KB)
Export: BibTeX | EndNote (RIS)

Abstract

The target tracking by unmanned aerial vehicle (UAV) has become a hot research topic in the field of computer vision.UAV target tracking can be applied to fire fighting, military and other important fields.At present, most UAV target tracking algorithms are based on traditional RGB cameras combined with deep learning algorithms. However, such methods are hard to deal with motion blur caused by UAV body jitter. Moreover, traditional RGB cameras have poor performance of imaging on low-illumination or over-exposure scenes. In order to solve the above problems, this paper presents an approach which takes the UAV with DAVIS event camera for target tracking, and designs a dual mode fusion tracking network based on events and grayscale, To better train the network, this paper employs the Vicon motion capture system to make the target tracking data sets:Event-APS 28 under the UAV perspective, ensuring that the target can be tracked effectively according to the image information even in complex lighting scenes.

Key words： target tracking event camera the dual-modal fusion tracking network

Received: 08 August 2021 Published: 18 January 2022

CLC:

TP 391.41

Corresponding Authors: Xiaopeng WEI,Xin YANG E-mail: xpwei@dlut.edu.cn;xinyang@dlut.edu.cn

	Service
	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	Qiang ZHU
	Chaoyi WANG
	Jiqing ZHANG
	Baocai YIN
	Xiaopeng WEI
	Xin YANG

Cite this article:

Qiang ZHU,Chaoyi WANG,Jiqing ZHANG,Baocai YIN,Xiaopeng WEI,Xin YANG. UAV target tracking algorithm based on event camera. Journal of Zhejiang University (Science Edition), 2022, 49(1): 10-18.

URL:

https://www.zjujournals.com/sci/EN/Y2022/V49/I1/10

基于事件相机的无人机目标跟踪算法

无人机目标跟踪可应用于消防、军事等重要领域，已成为计算机视觉领域热门研究课题之一。现有的无人机目标跟踪算法大多基于传统RGB相机结合深度学习算法，但此类算法一方面无法避免无人机机体抖动造成的运动模糊，另一方面因传统RGB相机在低光照或过曝光场景下成像质量较差，难以跟踪目标，为此提出采用无人机搭载DAVIS事件相机的方法进行目标跟踪。设计了基于事件与灰度图的双模态融合跟踪网络，用Vicon运动捕捉系统制作了无人机视角下的目标跟踪Event-APS 28数据集，实现了在复杂光照场景下对目标物的有效跟踪。

关键词： 目标跟踪, 事件相机, 双模态融合跟踪网络

Table 1 Comparison of Event-APS 28 dataset and other event datasets

Fig.1 Partial display of the Event-APS 28 dataset

Fig.2 Target tracking nework architecture of event and grayscale image dual-mode fusion

Fig.3 Network architecture of FFM

Fig.4 Network architecture of CA mechanism

Fig.5 Comparison of tracking of different algorithms

Fig.6 Comparative experimental of different algorithms

Table 2 Comparison of SR and PR


[1]	周小龙，刘倩倩，产思贤，等. 基于事件相机的视觉跟踪算法综述［J］. 小型微型计算机系统， 2020， 41（11）： 2325-2332. DOI：10.3969/j.issn.1000-1220.2020.11.015 ZHOU X L， LIU Q Q， CHAN S X， et al. Event camera-based visual tracking algorithms： A survey［J］. Journal of Chinese Computer Systems， 2020， 41（11）： 2325-2332. DOI：10.3969/j.issn.1000-1220. 2020. 11.015 doi: 10.3969/j.issn.1000-1220. 2020. 11.015

[2]	MOSSé Y P， LAUDENSLAGER M， LONGO L， et al. Identification of ALK as a major familial neuroblastoma predisposition gene［J］. Nature， 2008， 455（7215）： 930-935. DOI：10.1038/nature07261 doi: 10.1038/nature07261

[3]	刘勇. 一种改进CSK目标跟踪算法［J］. 电子技术与软件工程， 2018 （7）： 155-157. LIU Y. An improved CSK target tracking algorithm［J］. Electronic Technology & Software Engineering， 2018 （7）： 155-157.

[4]	周正松，陈虹君，周红. 基于多特征融合的尺度自适应KCF目标跟踪算法［J］. 四川大学学报（自然科学版）， 2020， 57（4）： 697-703. DOI：10.3969/j.issn. 0490-6756.2020.04.012 ZHOU Z S， CHEN H J， ZHOU H. Scale-adaptive kernel correlation filtering tracking algorithm based on multi-feature fusion［J］. Journal of Sichuan University（Natural Science Edition）， 2020， 57（4）： 697-703. DOI：10.3969/j.issn.0490-6756.2020.04.012 doi: 10.3969/j.issn.0490-6756.2020.04.012

[5]	BERTINETTO L， VALMADRE J， HENRIQUES J F， et al. Fully-convolutional siamese networks for object tracking［C］// European Conference on Computer Vision. Cham： Springer， 2016. doi:10.1007/978-3-319-48881-3_56 doi: 10.1007/978-3-319-48881-3_56

[6]	ZHENG W Y， ZHANG J. Feature extraction and image retrieval based on AlexNet［C］// Eighth International Conference on Digital Image Processing（ICDIP 2016）. Bellingham： International Society for Optics and Photonics， 2016. doi:10.1117/12.2243849 doi: 10.1117/12.2243849

[7]	BO L， YAN J J， WEI W， et al. High performance visual tracking with siamese region proposal network［C］// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE， 2018. doi:10.1109/cvpr.2018.00935 doi: 10.1109/cvpr.2018.00935

[8]	LIANG Z H， LIANG C J， ZHANG Y， et al. Tracking of moving target based on siammask for video SAR system［C］// 2019 IEEE International Conference on Signal， Information and Data Processing （ICSIDP）. Piscataway： IEEE， 2019. doi:10.1109/icsidp47821.2019.9173432 doi: 10.1109/icsidp47821.2019.9173432

[9]	林靖豪. 用于视频问答的多级注意力循环神经网络算法研究［D］. 杭州：浙江大学， 2019. doi:10.1007/s11063-019-10003-1 LIN J H. Multi-grained Hierarchical Attentional Recurrent Network for Video Question Answering［D］. Hangzhou： Zhejiang University， 2019. doi:10.1007/s11063-019-10003-1 doi: 10.1007/s11063-019-10003-1

[10]	ATTIA P， PHAN G Q， MAKER A V， et al. Autoimmunity correlates with tumor regression in patients with metastatic melanoma treated with anti-cytotoxic T-lymphocyte antigen-4［J］. Journal of Clinical Oncology， 2005， 23（25）：6043-6053. DOI：10.1200/JCO.2005.06.205 doi: 10.1200/JCO.2005.06.205

[11]	考夫曼，库尔茨，阿姆伯格，等. 基于事件相机的可变形对象跟踪CN111417983A［P］. 2020-07-24. doi:10.3969/j.issn.1000-7504.2008.06.001 KAUFMAN P， KURTZ D， AMBERG B， et al. Deformable Object Tracking Based on Event Camera CN111417983A［P］. 2020-07-24. doi:10.3969/j.issn.1000-7504.2008.06.001 doi: 10.3969/j.issn.1000-7504.2008.06.001

[12]	江盟，刘舟，余磊. 低维流形约束下的事件相机去噪算法［J］. 信号处理， 2019， 35（10）： 1753-1761. DOI：10.16798/j.issn.1003-0530.2019.10.017 JIANG M， LIU Z， YU L. A denoising algorithm for event cameras based on low-dimensional manifold constraint［J］. Journal of Signal Processing， 2019， 35（10）： 1753-1761. DOI：10.16798/j.issn.1003-0530. 2019.10.017 doi: 10.16798/j.issn.1003-0530. 2019.10.017

[13]	RUSSAKOVSKY O， DENG J， SU H， et al. ImageNet large scale visual recognition challenge［J］. International Journal of Computer Vision， 2015， 115（3）： 211-252. DOI：10.1007/s11263-015-0816-y doi: 10.1007/s11263-015-0816-y

[14]	王萌，王策，栗思思，等. 深度学习模型融合正则化方法在高维数据特征筛选中的应用研究［J］. 中国卫生统计， 2021， 38（1）： 73-75， 80. DOI：10.3969/j.issn. 1002-3674.2021.01.019 WANG M， WANG C， LI S S， et al. Application research of deep learning model fusion regularization method in high-dimensional data feature screening［J］. Chinese Journal of Health Statistics， 2021， 38（1）： 73-75， 80. DOI：10.3969/j.issn.1002-3674.2021.01.019 doi: 10.3969/j.issn.1002-3674.2021.01.019

[15]	程建，周越，蔡念，等.基于粒子滤波的红外目标跟踪［J］.红外与毫米波学报，2006，25（2）：113-117. doi:10.3321/j.issn:1001-9014.2006.02.008 CHEN J，ZHOU Y，CAI N，et al. Infrared object tracking based on particle filters［J］.Journal of Infrared Millimeter Waves，2006，25（2）：113-117. doi:10.3321/j.issn:1001-9014.2006.02.008 doi: 10.3321/j.issn:1001-9014.2006.02.008

[16]	JANG E， GU S X， POOLE B. Categorical reparameterization with gumbel-softmax［C］// The fifth International Conference on Learning Representations. Toulon：ICLR， 2017.

[17]	高帆，吴国平，刑晨，等. TLD目标跟踪算法研究［J］. 电视技术， 2013， 37（11）： 70-74， 202. DOI：10.3969/j.issn.1002-8692.2013.11.019 GAO F， WU G P， XING C， et al. TLD target tracking algorithm［J］. Video Engineering， 2013， 37（11）： 70-74， 202. DOI：10.3969/j.issn.1002-8692. 2013.11.019 doi: 10.3969/j.issn.1002-8692. 2013.11.019

[18]	LI Q， GE X S， WANG G. An improved TLD tracking method using compressive sensing［C］// Proceedings of the 2016 International Conference on Artificial Intelligence： Technologies & Applications.Paris： Atlantis Press， 2016. doi:10.2991/icaita-16.2016.64 doi: 10.2991/icaita-16.2016.64

[19]	MEADER J L， HART F L. CLNET：A computer model for tracking movement， decay， and concentrations throughout water distribution systems［C］// Critical Water Issues & Computer Applications. Reston： ASCE， 2015.

[20]	CHOI S， LEE J， LEE Y， et al. Robust long-term object tracking via improved discriminative model prediction［J］. ECCV 2020 Workshops. Cham： Springer， 2020. DOI：10.1007/978-3-030-68238-5_40 doi: 10.1007/978-3-030-68238-5_40

[21]	王海涛，王荣耀，王文皞，等. 目标跟踪综述［J］. 计算机测量与控制， 2020， 28（4）： 1-6，21. DOI：10. 16526/j.cnki.11-4762/tp.2020.04.001 WANG H T， WANG R Y， WANG W H， et al. A survey on recent advance and trends in object tracking［J］. Computer Measurement & Control， 2020， 28（4）： 1-6，21. DOI：10.16526/j.cnki.11-4762/tp.2020.04.001 doi: 10.16526/j.cnki.11-4762/tp.2020.04.001

[1]	Yuhua FANG,Feng YE. MFDC-Net: A breast cancer pathological image classification algorithm incorporating multi-scale feature fusion and attention mechanism[J]. Journal of Zhejiang University (Science Edition), 2023, 50(4): 455-464.

[2]	Ruiqi YU,Yuhua LIU,Xilong SHEN,Ruyu ZHAI,Xiang ZHANG,Zhiguang ZHOU. Representation learning driven multiple graph sampling[J]. Journal of Zhejiang University (Science Edition), 2022, 49(3): 271-279.

[3]	Jintai ZHU,Jihua YE,Feng GUO,Lu JIANG,Aiwen JIANG. FSAGN:An expression recognition method based on independent selection of video key frames[J]. Journal of Zhejiang University (Science Edition), 2022, 49(2): 141-150.

[4]	Ying ZHONG,Song WANG,Hao WU,Zepeng CHENG,Xuejun LI. SEMMA-Based visual exploration of cyber security event[J]. Journal of Zhejiang University (Science Edition), 2022, 49(2): 131-140.

[5]	Meng YANG,Shu DING,Yuntao MA,Jiayi XIE,Ruifeng DUAN. Dynamic simulation method of wheat rust based on texture feature[J]. Journal of Zhejiang University (Science Edition), 2022, 49(1): 1-9.

[6]	YU Peng, LIU Lan, CAI Yun, HE Yu, ZHANG Songhai. Home fitness monitoring system based on monocular camera[J]. Journal of Zhejiang University (Science Edition), 2021, 48(5): 521-530.

[7]	FU Rujia, XIAN Chuhua, LI Guiqing, WAN Juanjie, CAO Cheng, YANG Cunyi, GAO Yuefang. Rapid 3D reconstruction of bean plant for accurate phenotype identification[J]. Journal of Zhejiang University (Science Edition), 2021, 48(5): 531-539.

[8]	GUI Zhiqiang, YAO Yuyou, ZHANG Gaofeng, XU Benzhu, ZHENG Liping. An efficient computation method of 3D-power diagram[J]. Journal of Zhejiang University (Science Edition), 2021, 48(4): 410-417.

[9]	XU Min, WANG Ke, DAI Haoran, LUO Xiaobo, YU Weilun, TAO Yubo, LIN Hai. Visual analysis of cohorts and treatments of breast cancer based on electronic health records[J]. Journal of Zhejiang University (Science Edition), 2021, 48(4): 391-401.

[10]	ZOU Beiji, YANG Wenjun, LIU Shu, JIANG Lingzi. A three-stage text recognition framework for natural scene images[J]. Journal of Zhejiang University (Science Edition), 2021, 48(1): 1-8.

[11]	CHEN Yuanqiong, ZOU Beiji, ZHANG Meihua, LIAO Wangmin, HUANG Jiaer, ZHU Chengzhang. A review on deep learning interpretability in medical image processing[J]. Journal of Zhejiang University (Science Edition), 2021, 48(1): 18-29.

[12]	DENG Huijun. Ranking-supported interactive data classification method and its application[J]. Journal of Zhejiang University (Science Edition), 2021, 48(1): 9-17.

[13]	LI Huabiao, HOU Xiaogang, WANG Tingting, ZHAO Haiying. An unified generation scheme of traditional patterns based on rule learning[J]. Journal of Zhejiang University (Science Edition), 2020, 47(6): 669-676.

[14]	TAN Jieqing, CAO Ningning. A new Midedge scheme of quadrilateral mesh[J]. Journal of Zhejiang University (Science Edition), 2019, 46(2): 154-163.

Viewed

Full text

Abstract

Cited

Shared

Discussed