Please wait a minute...
Journal of ZheJiang University (Engineering Science)  2023, Vol. 57 Issue (8): 1516-1526    DOI: 10.3785/j.issn.1008-973X.2023.08.005
    
Lightweight YOLOv5s network-based algorithm for identifying hazardous objects under vehicles
Xin JIN(),Jian-jun ZHUANG*(),Zi-heng XU
School of Electronics and Information Engineering, Nanjing University of Information Science and Technology, Nanjing 210044, China
Download: HTML     PDF(3082KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

A lightweight SG-YOLOv5s network model was proposed to solve the problems of complex structure, large number of parameters and difficult deployment on the end side of the current detection model of hazardous objects under vehicle. Firstly, the backbone and the neck of YOLOv5s network were optimized and improved, which significantly reduced the number of network parameters and greatly reduced the weight volume of the model. Secondly, in the training stage, Mixup data was used to enhance the generalization ability of the model. Finally, SIoU was used to replace the frame regression loss function CIoU, so that the hazardous object prediction box was closer to the real box and the detection accuracy was improved. In view of the fact that there were few data sets of hazardous objects under vehicle, a large number of fragmented images of car undersides were captured using an intelligent small car, and the AutoStitch algorithm was adopted to splice the images, and finally the self-built data set of car bottom images was obtained. Experimental results show that, the identification accuracy rate of SG-YOLOv5s model was 97.63% in the self-built data sets of nine simulated vehicle hazards, which was 1.26% higher than that of the original YOLOv5s model. Additionally, the SG-YOLOv5s model reduced the number of parameters by 71.27% and decreased the model weight volume by 71.28%. These advancements provide the potential for embedded deployment of subsequent recognition models.



Key wordslightweight model      YOLOv5s      data enhancement      target recognition      mosaic of images     
Received: 01 December 2022      Published: 31 August 2023
CLC:  TP 391.4  
Fund:  国家重点研发计划资助项目(2021YFE0105500);国家自然科学基金资助项目(62171228 );江苏高校‘青蓝工程’资助项目
Corresponding Authors: Jian-jun ZHUANG     E-mail: 1278491940@qq.com;jjzhuang@nuist.edu.cn
Cite this article:

Xin JIN,Jian-jun ZHUANG,Zi-heng XU. Lightweight YOLOv5s network-based algorithm for identifying hazardous objects under vehicles. Journal of ZheJiang University (Engineering Science), 2023, 57(8): 1516-1526.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2023.08.005     OR     https://www.zjujournals.com/eng/Y2023/V57/I8/1516


轻量化YOLOv5s网络车底危险物识别算法

针对现有车底危险物检测模型结构复杂、参数量大、不易部署于端侧的问题,提出轻量化SG-YOLOv5s网络模型. 对YOLOv5s网络的骨干和颈部进行优化改进,显著降低网络的参数量,大幅缩小模型的权重体积;在训练阶段采用Mixup数据增强,提高模型的泛化能力;采用SIoU替换边框回归损失函数CIoU,使危险物预测框更接近真实框,提高检测精度. 鉴于车底危险物数据集较少的现状,利用智能小车拍摄大量车底碎片化图像,采用AutoStitch算法进行图像拼接,最终获得自建车底图像数据集. 实验结果表明:在自建的9种模拟车底危险物数据集上,SG-YOLOv5s模型识别精确率为97.63%,相较于原YOLOv5s模型提升了1.26%,而参数量减少了71.27%,模型权重体积下降了71.28%,为后续识别模型的嵌入式部署提供了可能.


关键词: 轻量化模型,  YOLOv5s,  数据增强,  目标识别,  图像拼接 
Fig.1 YOLOv5s network structure
Fig.2 Mosaic data enhancement
Fig.3 Structure of Focus, CBS, CSP and SPPF
Fig.4 SG-YOLOv5s network structure
Fig.5 Basic unit of ShuffleNet v2
序号 模块重复次数 模块名 参数配置 输出大小
0 1 Focus [3, 32, 3] 32×320×320
1 1 S-(2) [32, 64, 2] 64×160×160
2 1 S-(1) [64, 64, 1] 64×160×160
3 1 S-(2) [64, 128, 2] 128×80×80
4 3 S-(1) [128, 128, 1] 128×80×80
5 1 S-(2) [128, 256, 2] 256×40×40
6 3 S-(1) [256, 256, 1] 256×40×40
7 1 S-(2) [256, 512, 2] 512×20×20
8 1 SPPF [512, 512, [5, 5, 5]] 512×20×20
9 1 S-(1) [512, 512, 1] 512×20×20
Tab.1 Structure parameters of Backbone network
Fig.6 Ghost convolutional module
Fig.7 Overall structure of GhostBottleneck
Fig.8 Mixup data enhancement
Fig.9 Schematic diagram of angle cost
Fig.10 Fragmented image of a car bottom
Fig.11 Complete image of hazardous objects under vehicle
类别 AP/% 类别 AP/%
手套 99.76 袋子 99.33
剪刀 94.69 棍子 98.16
老虎钳 100.00 滚筒 97.05
塑料瓶 98.28 螺丝刀 99.19
92.18
Tab.2 Average accuracy of each type of hazardous object
Fig.12 Detection of different types of hazardous objects under different types of vehicles
模型 P/% R/% Par/MB Me/MB FPS mAP_0.5/%
①YOLOv5s 96.59 94.08 7.03 26.81 50.39 96.37
②YOLOV5s+Mixup 97.60 94.31 7.03 26.81 49.22 96.49
③YOLOv5s+Backbone+Mixup 96.17 94.50 4.15 15.83 46.37 96.49
④YOLOv5s+Neck+Mixup 96.99 96.96 4.90 18.68 45.33 97.00
⑤Backbone+Neck+Mixup+CIoU 96.92 97.02 2.02 7.70 47.04 97.19
⑥Backbone+Neck+Mixup+SIoU 96.80 97.96 2.02 7.70 47.39 97.63
Tab.3 Analysis of ablative experimental results for SG-YOLOv5s network model
尺寸/像素 mAP_0.5/% FPS
320×320 79.95 35.53
416×416 87.58 38.99
512×512 92.99 42.62
640×640 97.63 47.39
Tab.4 Performance comparison of SG-YOLOv5s network model at different image sizes
模型 P/% R/% Par/MB Me/MB FPS mAP_0.5/%
Faster R-CNN 71.51 85.61 137.10 522.99 13.18 87.88
YOLOv3 93.63 87.39 61.53 234.74 39.63 92.11
YOLOv4 93.82 91.22 63.95 243.94 30.97 93.97
YOLOX-s 97.74 97.21 8.94 34.10 42.89 97.15
YOLOv7 96.75 94.97 37.21 141.93 33.14 97.03
YOLOv5s 96.59 94.08 7.03 26.81 50.39 96.37
SG-YOLOv5s 96.80 97.96 2.02 7.70 47.39 97.63
Tab.5 Performance comparison of common object detection models
Fig.13 Comparison of detection effects between SG-YOLOv5s model and original YOLOv5s model
[1]   张辉, 李树君, 王伟平, 等 视频车底安检系统的开发[J]. 工业仪表与自动化装置, 2010, (2): 27- 29
ZHANG Hui, LI Shu-jun, Wang Wei-ping, et al Development of under vehicle surveillance system[J]. Industrial Instrumentation and Automation, 2010, (2): 27- 29
doi: 10.3969/j.issn.1000-0682.2010.02.007
[2]   徐常星, 郭守学 智能视觉车底安检便携移动平台[J]. 中国安全防范技术与应用, 2018, (6): 6- 10
XU Chang-xing, GUO Shou-xue Intelligent vision under car security portable mobile platform[J]. China Security Protection Technology and Application, 2018, (6): 6- 10
[3]   许金金, 王敏, 吴伟交 基于红外图像的车底藏人检测系统设计[J]. 计算机工程与应用, 2013, 49 (3): 69- 73
XU Jin-jin, WANG Min, WU Wei-jiao Infrared image-based system for detecting hiding personnel under vehicle[J]. Computer Engineering and Applications, 2013, 49 (3): 69- 73
doi: 10.3778/j.issn.1002-8331.1206-0027
[4]   徐哲伟, 刘昭, 包建东, 等 基于改进YoloX-s的密贴检查器故障检测方法[J]. 电子测量技术, 2022, 45 (12): 91- 99
XU Zhe-wei, LIU Zhao, BAO Jian-dong, et al A fault detection method of closure detectors based on the improved YoloX-s[J]. Electronic Measurement Technology, 2022, 45 (12): 91- 99
[5]   高春艳, 赵文辉, 张明路, 等 一种基于YOLOv3的汽车底部危险目标检测算法[J]. 天津大学学报: 自然科学与工程技术版, 2020, 53 (4): 358- 365
GAO Chun-yan, ZHAO Wen-hui, ZHANG Ming-lu, et al A vehicle bottom dangerous object detection algorithm based on YOLOv3[J]. Journal of Tianjin University: Science and Technology, 2020, 53 (4): 358- 365
[6]   REDMON J, FARHADI A. YOLOv3: an incremental improvement [EB/OL]. [2018-04-08]. https://arxiv.org/abs/1804.02767.
[7]   黄锦波. 基于AutoStitch的无人机航拍图像拼接技术研究[D]. 西安: 西安电子科技大学, 2020.
HUANG Jin-bo. Research on UAV aerial image stitching based on AutoStitch [D]. Xi'an: Xidian University, 2020.
[8]   ZHANG H, CISSE M, DAUPHIN Y N, et al. Mixup: beyond empirical risk minimization[EB/OL]. [2021-02-10]. https://doi.org/10.48550/arXiv.1710.09412.
[9]   ZHENG Z, WANG P, LIU W, et al. Distance-IoU loss: faster and better learning for bounding box regression[C]// AAAI Conference on Artificial Intelligence. New York: AAAI, 2020: 12993-13000.
[10]   GEVORGYAN Z. SIoU loss: more powerful learning for bounding box regression [EB/OL]. [2022-05-25]. https://arxiv.org/abs/2205.12740.
[11]   ULTRALYTICS. YOLOv5[EB/OL]. [2021-4-15]. https://github. com/ultralytics/yolov5.
[12]   王一, 龚肖杰, 程佳, 等 基于改进YOLOv5的金属工件表面缺陷检测[J]. 包装工程, 2022, 43 (15): 54- 60
WANG Yi, GONG Xiao-jie, CHENG Jia, et al Surface defect detection of metal workpiece based on improved YOLOv5[J]. Packaging Engineering, 2022, 43 (15): 54- 60
[13]   LIN T Y, DOLLAR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu: IEEE, 2017: 2117-2125.
[14]   LIU S, QI L, QIN H, et al. Path aggregation network for instance segmentation[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Salt Lake City: IEEE, 2018: 8759-8768.
[15]   李小波, 李阳贵, 郭宁, 等 融合注意力机制的 YOLOv5 口罩检测算法[J]. 图学学报, 2023, 44 (1): 16- 25
LI Xiao-bo , LI Yang-gui, GUO Ning, et al Mask detection algorithm based on YOLOv5 integrating attention mechanism[J]. Journal of Graphics, 2023, 44 (1): 16- 25
[16]   彭继慎, 孙礼鑫, 王凯, 等 基于模型压缩的ED-YOLO电力巡检无人机避障目标检测算法[J]. 仪器仪表学报, 2021, 42 (10): 161- 170
PENG Ji-shen, SUN Li-xin, WANG Kai, et al ED-YOLO power inspection UAV obstacle avoidance target detection algorithm based on model compression[J]. Chinese Journal of Scientific Instrument, 2021, 42 (10): 161- 170
[17]   MA N, ZHANG X, ZHENG H T, et al. ShuffleNet V2: practical guidelines for efficient CNN architecture design [C]// European Conference on Computer Vision. Salt Lake City: CVPR, 2018.
[18]   史梦圆, 高俊钗 改进YOLOv5的建筑物破损检测算法研究[J]. 国外电子测量技术, 2022, 41 (8): 84- 90
SHI Meng-yuan, GAO Jun-chai Research on improved YOLO v5 building damage detection algorithm[J]. Foreign Electronic Measurement Technology, 2022, 41 (8): 84- 90
[19]   HAN K, WANG Y, TIAN Q, et al. GhostNet: more features from cheap operations[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle: IEEE, 2020.
[20]   ZHOU D, FANG J, SONG X, et al. IoU loss for 2D/3D object detection [C]// 2019 International Conference on 3D Vision (3DV). Quebec City: IEEE, 2019: 85-94.
[21]   刘洪江, 曹玉香, 李佳, 等 基于Opencv图像处理的无人机影像拼接技术研究[J]. 城市勘测, 2021, (2): 83- 86
LIU Hong-jiang, CAO Yu-xiang, LI Jia, et al Research on UAV image Mosaic technology based on Opencv image processing[J]. Urban Geotechnical Investigation and Surveying, 2021, (2): 83- 86
[22]   徐印赟, 江明, 李云飞, 等. 基于改进YOLO及NMS的水果目标检测[J]. 电子测量与仪器学报, 2022, 36(4): 114-123.
XU Yin-yun, JIANG Ming, LI Yun-fei, et al. Fruit target detection based on improved YOLO and NMS [J]. Journal of Electronic Measurement and Instrumentation, 2022, 36(4): 114-123.
[23]   REN S, HE K, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2017, 39(6): 1137-1149.
[24]   BOCHKOVSKIY A, WANG C Y, LIAO H Y M. YOLOv4: optimal speed and accuracy of object detection[EB/OL]. [2020-04-23]. https://arxiv.org/abs/2004.10934.
[25]   GE Z, LIU S, WANG F, et al. Yolox: exceeding yolo series in 2021[EB/OL]. [2021-07-18]. https://arxiv.org/abs/2107.08430.
[1] Qing-lu MA,Jia-ping LU,Xiao-yao TANG,Xue-feng DUAN. Improved YOLOv5s flame and smoke detection method in road tunnels[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(4): 784-794.
[2] Kun HAO,Kuo WANG,Bei-bei WANG. Lightweight underwater biological detection algorithm based on improved Mobilenet-YOLOv3[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(8): 1622-1632.
[3] Yun-zuo ZHANG,Wei GUO,Zhao-quan CAI,Wen-bo LI. Remote sensing image target detection combining multi-scale and attention mechanism[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(11): 2215-2223.
[4] Kun LIU,Xi WEN,Min-ming HUANG,Xin-xin YANG,Jing-kun MAO. Solar cell defect enhancement method based on generative adversarial network[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(4): 684-693.