Please wait a minute...
浙江大学学报(工学版)
Journal of ZheJiang University (Engineering Science)  2023, Vol. 57 Issue (6): 1224-1233    DOI: 10.3785/j.issn.1008-973X.2023.06.018
    
UAV dense small target detection algorithm based on YOLOv5s
Jun HAN(),Xiao-ping YUAN*(),Zhun WANG,Ye CHEN
School of Information and Control Engineering, China University of Mining and Technology, Xuzhou 221116, China
Download: HTML     PDF(2789KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

The dense small target detection algorithm LSA_YOLO based on YOLOv5s for UAVs with complex backgrounds and multiples of small targets with dense distribution was proposed for UAV images. A multi-scale feature extraction module LM-fem was constructed to enhance the feature extraction capability of the network. A new hybrid domain attention module S-ECA relying on multi-scale contextual information has been put forward and a algorithm focus on target information was established aiming to suppress the interference of complex backgrounds. The adaptive weight dynamic fusion structure AFF was designed to assign reasonable fusion weights to both shallow and deep features. The capability of algorithm in detecting dense small targets in complex backgrounds was improved given the application of S-ECA and AFF in the structure of PANet. The loss function Focal-EIOU was utilized instead of the loss function CIOU to accelerate model detection efficiency. Experimental results on the public dataset VisDrone2021 public dataset show that the average detection accuracy for all target classes improves from 51.5% for YOLOv5s to 57.6% for LSA_YOLO when the set input resolution is set to 1 504 × 1 504.

Key words： UAV      small target detection      multi-scale features      attention mechanism      feature fusion     
Received: 24 June 2022      Published: 30 June 2023
CLC:  V 279  
  TP 391.4  
  TP 183  
Fund:  国家科技支撑计划资助项目（2013BAK06B08）； 国家自然科学基金资助项目（32171241）
Corresponding Authors: Xiao-ping YUAN     E-mail: m19816250697@163.com;1941@cumt.edu.cn
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Jun HAN
Xiao-ping YUAN
Zhun WANG
Ye CHEN
Cite this article:

Jun HAN,Xiao-ping YUAN,Zhun WANG,Ye CHEN. UAV dense small target detection algorithm based on YOLOv5s. Journal of ZheJiang University (Engineering Science), 2023, 57(6): 1224-1233.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2023.06.018     OR     https://www.zjujournals.com/eng/Y2023/V57/I6/1224


基于YOLOv5s的无人机密集小目标检测算法

针对无人机图像中背景复杂、小目标数量多且分布密集的特点，提出基于YOLOv5s的无人机密集小目标检测算法LSA_YOLO. 构造多尺度特征提取模块LM-fem，增强网络的特征提取能力. 为了抑制复杂背景的干扰，使算法关注目标信息，提出依靠多尺度上下文信息的、新的混合域注意力模块S-ECA. 设计自适应权重动态融合结构AFF，为浅层特征和深层特征合理分配融合权重. 将S-ECA、AFF应用于PANet结构，提高算法在复杂背景下的密集小目标检测能力. 使用损失函数Focal-EIOU代替损失函数CIOU，增强模型检测性能. 在公开数据集VisDrone2021上的实验结果表明，当设置输入分辨率为1 504 $ \times $1 504时，对所有目标类别的平均检测精度从YOLOv5s的51.5%提高到LSA_YOLO的57.6%.


关键词： 无人机,  小目标检测,  多尺度特征,  注意力机制,  特征融合 
Fig.1 Architecture of LSA_YOLO algorithm
Fig.2 Structure comparison of LM-fem and bottleneck
Fig.3 Architecture of squeeze-and-excitation module
Fig.4 Architecture of spatial location attention module
Fig.5 Architecture of mixed domain attention module
Fig.6 Architecture of adaptive weight dynamic fusion module
Fig.7 Architecture of improved PANet
Fig.8 Data information in VisDrone2021 dataset
编号 模型 mAP $ {}_{50} $/% mAP $ {}_{50:95} $/% NP/106 O F/(帧·s?1) M/% E/%
1 YOLOv5s 33.2 16.7 6.978 15.5 125 57.5 47.2
2 YOLOv5s+LM-fem 33.9 17.5 6.920 14.9 130 55.0 45.3
3 YOLOv5s+CBAMbackbone 34.7 18.5 7.556 17.5 98 54.2 42.8
4 YOLOv5s+S-ECAbackbone 35.9 20.4 7.540 17.2 105 52.5 39.5
5 YOLOv5s+AFFneck 37.1 22.3 7.015 15.9 111 50.2 37.0
6 YOLOv5s+ $ \mathrm{S}\mathrm{A}\_\mathrm{P}\mathrm{A}\mathrm{N}\mathrm{e}\mathrm{t} $neck 37.8 22.9 8.135 18.4 95 48.9 35.2
7 YOLOv5s+Focal-EIOU[15] 33.5 17.2 7.322 16.1 120 56.6 45.8
8 LSA_YOLO 41.1 25.5 9.038 20.2 50 45.7 31.5
Tab.1 Evaluation index of detection performance of each model in ablation experiment
Fig.9 Mean average precision for each model in ablation experiment
模型 mAP ${}_{50}/\text{%}$ NP/106 O F/(帧·s? 1)
$ \mathrm{Y}\mathrm{O}\mathrm{L}\mathrm{O}\mathrm{v}5\mathrm{s} $640 33.2 6.978 15.5 125
$ \mathrm{Y}\mathrm{O}\mathrm{L}\mathrm{O}\mathrm{v}5\mathrm{s} $1024 47.0 6.978 41.0 125
$ \mathrm{Y}\mathrm{O}\mathrm{L}\mathrm{O}\mathrm{v}5\mathrm{s} $1504 51.5 6.978 88.2 125
LSA_YOLO640 41.1 9.038 20.2 50
LSA_YOLO1024 49.7 9.038 51.7 50
LSA_YOLO1504 57.6 9.038 110.3 50
Tab.2 Evaluation indicators for detection performance of models at different resolutions
算法 AP/% mAP $ {}_{50} $/%
A B C D E F G H I J
TridentNet[16] 22.8 9.0 5.3 46.2 30.7 25.5 21.3 16.0 39.0 17.9 43.1
RRNet[17] 30.5 14.8 14.1 51.5 35.8 35.2 28.8 19.0 45.0 26.0 55.0
CenterNet[18] 28.0 12.0 8.9 51.2 35.9 27.5 21.0 19.8 37.7 20.9 48.5
YOLOv5+head — — — — — — — — — — 33.8
YOLOv5+upsampling — — — — — — — — — — 50.5
YOLOv5+ M-Bi — — — — — — — — — — 43.6
YOLOv4[19] 25.0 13.1 8.5 64.2 22.5 22.6 11.5 8.0 44.5 22.0 43.0
YOLOv3-LITE[20] 34.6 22.9 8.0 71.2 31.4 22.1 15.5 7.1 41.3 32.7 41.9
MSC-CenterNet[21] 33.5 15.3 12.5 55.2 40.6 32.0 29.2 21.6 42.5 27.4 39.5
Faster R-CNN[22] 21.0 14.7 7.5 51.0 30.2 19.6 15.7 9.5 31.6 20.3 33.2
LSA_YOLO 37.2 25.4 18.5 58.6 35.7 35.8 29.4 21.5 47.2 28.4 57.6
Tab.3 Average precision and mean average precision for different algorithms on VisDrone2021 dataset
Fig.10 Bar chart of mean average precision for different algorithms on VisDrone2021 dataset
Fig.12 Detection effect of LSA_YOLO algorithm in complex scenes
Fig.11 Visual comparison of target detection results for different algorithms
Fig.13 Comparison of detection effects between LSA_YOLO algorithm and baseline algorithm
[1]   奉志强, 谢志军, 包正伟, 等 基于改进 YOLOv5的无人机实时密集小目标检测算法[J]. 航空学报, 2023, 44 (3): 327106
FENG Zhi-qiang, XIE Zhi-jun, BAO Zheng-wei, et al Real-time dense small object detection algorithm for UAV based on improved YOLOv5[J]. Acta Aeronautica et Astronautica Sinica, 2023, 44 (3): 327106
[2]   REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 779-788.
[3]   REDMON J, FARHADI A. YOLO9000: better, faster, stronger [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 6517-6525.
[4]   REDMON J, FARHADI A. YOLOv3: an incremental improvement [EB/OL]. [2022-06-20]. https://arxiv.org/abs/1804.02767.
[5]   BOCHKOVSKIY A, WANG C Y, LIA O H. YOLOv4: optimal speed and accuracy of object detection [EB/OL]. [2022-06-20]. https://arxiv.org/abs/2004.10934.
[6]   LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot multibox detector [C]// European Conference on Computer Vision. [S.l.]: Springer, 2016: 21-37.
[7]   李科岑, 王晓强, 林浩, 等 深度学习中的单阶段小目标检测方法综述[J]. 计算机科学与探索, 2022, 16 (1): 41- 58
LI Ke-cen, WANG Xiao-qiang, LIN Hao, et al Survey of one-stage small object detection methods in deep learning[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16 (1): 41- 58
[8]   XIE L C, XUE Y L, YE J Z. UAV aerial photography target detection algorithm based on improved YOLOv5 [C]// Journal of Physics: Conference Series. [S.l.]: IOP Publishing, 2022, 2284(1): 012024.
[9]   YANG Y Z. Drone-view object detection based on the improved YOLOv5 [C]// Proceedings of the IEEE International Conference on Electrical Engineering, Big Data and Algorithms. Changchun: IEEE, 2022: 612-617.
[10]   吴萌萌, 张泽斌, 宋尧哲, 等. 基于自适应特征增强的小目标检测网络[J/OL]. 激光与光电子学进展, 2023, 60(6): 0610004. [20222-06-20]. https://www.opticsjournal.net/Articles/OJ7e6f90484b1776fd/References.
WU Meng-meng, ZHANG Ze-bin, SONG Yao-zhe, et al. Small object detection network based on adaptive feature enhancement [J/OL]. Advances in Laser and Opt-oelectronics, 2023, 60(66): 0610004. [2022-06-20]. https://www.opticsjournal.net/Articles/OJ7e6f90484b1776fd/References.
[11]   HU J, SHEN L, SUN G. Squeeze-and-excitation networks [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 7132-7141.
[12]   LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 2117-2125.
[13]   LI H, XIONG P, AN J, et al. Pyramid attention network for semantic segmentation [EB/OL]. [2022-06-20]. https://arxiv.org/abs/1805.10180.
[14]   LIU S, QI L, QIN H F, et al. Path aggregation network for instance segmentation [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 8759–8768.
[15]   ZHANG Y F, REN W, ZHANG Z, et al Focal and efficient IOU loss for accurate bounding box regression[J]. Neurocomputing, 2022, 506: 146- 157
doi: 10.1016/j.neucom.2022.07.042
[16]   LI Y , CHEN Y , WANG N , et al. Scale-aware trident networks for object detection [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 6054-6063.
[17]   CHEN C, ZHANG Y, LV Q, et al. RRNet: a hybrid detector for object detection in drone-captured images [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision Workshop. Seoul: IEEE, 2019: 100-108.
[18]   ZHOU X, WANG D, KRAHENBUHL P. Objects as points [EB/OL]. [2022-06-20]. https://arxiv.org/abs/1904.07850.
[19]   ALI S, SIDDIQUE A, ATES H F, et al. Improved YOLOv4 for aerial object detection [C]// Proceedings of the 29th Signal Processing and Communications Applications Conference. Istanbul: IEEE, 2021: 1-4.
[20]   ZHAO H, ZHOU Y, ZHANG L, et al Mixed YOLOv3-LITE: a lightweight real-time object detection method[J]. Sensors, 2020, 20 (7): 1861
doi: 10.3390/s20071861
[21]   DU D W, WEN L Y, ZHU P F, et al. VisDrone-DET2020: the vision meets drone object detection in image challenge results [C]// Proceedings of the European Conference on Computer Vision. [S.l.]: Springer, 2020: 692-712.
[1] Xue-yong XIANG,Li WANG,Wen-peng ZONG,Guang-yun LI. Point cloud instance segmentation based on attention mechanism KNN and ASIS module[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(5): 875-882.
[2] Jian-zhao ZHANG,Ji-chang GUO,Yu-dong WANG. Underwater image enhancement algorithm via fusing reverse medium transmission map[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(5): 921-929.
[3] Qing-lin AI,Jing-rui CUI,Bing-hai LV,Tong TONG. Surface defect detection method for bearing drum-shaped rollers based on fusion transformation of defective area[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(5): 1009-1020.
[4] Yu-ting SU,Rong-xuan LU,Wei ZHANG. Vehicle re-identification algorithm based on attention mechanism and adaptive weight[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(4): 712-718.
[5] Bai-cheng BIAN,Tian CHEN,Ru-jun WU,Jun LIU. Improved YOLOv3-based defect detection algorithm for printed circuit board[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(4): 735-743.
[6] Yan-fen CHENG,Jia-jun WU,Fan HE. Aspect level sentiment analysis based on relation gated graph convolutional network[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(3): 437-445.
[7] Fan YANG,Bo NING,Huai-qing LI,Xin ZHOU,Guan-yu LI. Multimodal image retrieval model based on semantic-enhanced feature fusion[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(2): 252-258.
[8] Chao LIU,Bing KONG,Guo-wang DU,Li-hua ZHOU,Hong-mei CHEN,Chong-ming BAO. Deep clustering via high-order mutual information maximization and pseudo-label guidance[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(2): 299-309.
[9] Lin-tao WANG,Qi MAO. Position measurement method for tunnel segment grabbing based on RGB and depth information fusion[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(1): 47-54.
[10] Li-zhou FENG,Yang YANG,You-wei WANG,Gui-jun YANG. New method for news recommendation based on Transformer and knowledge graph[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(1): 133-143.
[11] Hai-jun WANG,Sheng-yan ZHANG,Yu-jie DU. UAV object tracking algorithm based on response and filter deviation-aware regularization[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(9): 1824-1832.
[12] Kun HAO,Kuo WANG,Bei-bei WANG. Lightweight underwater biological detection algorithm based on improved Mobilenet-YOLOv3[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(8): 1622-1632.
[13] Ren-peng MO,Xiao-sheng SI,Tian-mei LI,Xu ZHU. Bearing life prediction based on multi-scale features and attention mechanism[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(7): 1447-1456.
[14] Xia HUA,Xin-qing WANG,Ting RUI,Fa-ming SHAO,Dong WANG. Vision-driven end-to-end maneuvering object tracking of UAV[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(7): 1464-1472.
[15] You-wei WANG,Shuang TONG,Li-zhou FENG,Jian-ming ZHU,Yang LI,Fu CHEN. New inductive microblog rumor detection method based on graph convolutional network[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(5): 956-966.