Please wait a minute...
浙江大学学报(工学版)  2023, Vol. 57 Issue (4): 784-794    DOI: 10.3785/j.issn.1008-973X.2023.04.016
交通工程、土木工程     
改进YOLOv5s的公路隧道烟火检测方法
马庆禄1,2(),鲁佳萍1,唐小垚1,段学锋3
1. 重庆交通大学 交通运输学院,重庆 400074
2. 山区复杂道路环境“人-车-路”协同与安全重庆市重点实验室,重庆 400074
3. 宁夏交投高速公路管理有限公司,宁夏 银川 750000
Improved YOLOv5s flame and smoke detection method in road tunnels
Qing-lu MA1,2(),Jia-ping LU1,Xiao-yao TANG1,Xue-feng DUAN3
1. School of Traffic and Transportation, Chongqing Jiaotong University, Chongqing 400074, China
2. Chongqing Key Laboratory of "Human-Vehicle-Road" Cooperation and Safety for Mountain Complex Environment, Chongqing 400074, China
3. Ningxia Jiaotou Expressway Management Limited Company, Yinchuan 750000, China
 全文: PDF(3978 KB)   HTML
摘要:

针对公路隧道初期火灾烟火混淆且检测实时性要求高的问题,提出改进YOLOv5s的隧道烟火视觉检测方法. 该方法通过在YOLOv5s中引入卷积注意力模块(CBAM),提高对轮廓特征不明显的隧道烟雾及初期火焰重要特征检测的准确率. 替换骨干网络中的Focus模块,降低BottleneckCSP的卷积层数目,提升烟火特征提取网络效率. 用CIoU替换原有的GIoU损失函数,加快模型的收敛速度. 实验以10 000张隧道烟火数据集为训练样本,用YOLOv5s和改进后的YOLOv5s-PRO进行对比试验分析,用2021年3月6日重庆真武山隧道火灾视频数据验证模型. 实验结果表明,该算法的检测精度达到91.53%,比YOLOv5s提高了3.21%,检测速度达到6.12 ms,比YOLOv5s提高了0.42 ms,检测精度较高,速度较快,可以应用于实际公路隧道的烟火检测.

关键词: 隧道工程隧道烟火检测注意力模块深度学习YOLOv5s    
Abstract:

An improved YOLOv5s for visual detection of smoke and fire in early-stage road tunnel fires was proposed to solve the problem of smoke and fire confusion and the requirement for real-time detection. The convolutional block attention module (CBAM) was introduced into YOLOv5s to improve the accuracy of detecting smoke with obscure contour features and initial tunnel flame with crucial features. The Focus module in the backbone network was replaced, the number of convolutional layers in BottleneckCSP was reduced, and the efficiency of the smoke and flame feature extraction network was improved. The CIoU was used to replace the original GIoU loss function to accelerate the convergence rate of the model. A data set containing 10 000 images of tunnel smoke and flame was used as the training sample. YOLOv5s and improved YOLOv5s-PRO were used for comparative test analysis. The model was validated by using the video data of the Zhenwu Mountain tunnel fire that occurred on March 6, 2021, in Chongqing, China. The experimental results showed that the detection accuracy of the algorithm reached up to 91.53%, which was 3.21% higher than YOLOv5s, and the detection speed reached 6.12 ms, which was 0.42 ms better than YOLOv5s. The YOLOv5s-PRO has higher detection accuracy and a faster rate, which can be applied to smoke and flame detection of actual road tunnel.

Key words: tunnel engineering    tunnel flame and smoke detection    attention module    deep learning    YOLOv5s
收稿日期: 2022-04-03 出版日期: 2023-04-21
CLC:  U 459  
基金资助: 国家社会科学基金资助项目(20VYJ023);宁夏回族自治区交通运输厅科技资助项目(NJGF20200301)
作者简介: 马庆禄(1980—),男,教授,博士,从事智能交通与安全研究. orcid.org/0000-0003-2641-0924. E-mail: mql@cqu.edu.cn
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章  
马庆禄
鲁佳萍
唐小垚
段学锋

引用本文:

马庆禄,鲁佳萍,唐小垚,段学锋. 改进YOLOv5s的公路隧道烟火检测方法[J]. 浙江大学学报(工学版), 2023, 57(4): 784-794.

Qing-lu MA,Jia-ping LU,Xiao-yao TANG,Xue-feng DUAN. Improved YOLOv5s flame and smoke detection method in road tunnels. Journal of ZheJiang University (Engineering Science), 2023, 57(4): 784-794.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2023.04.016        https://www.zjujournals.com/eng/CN/Y2023/V57/I4/784

图 1  YOLOv5s网络结构
图 2  CBAM网络的结构图
图 3  替换结构的示意图
图 4  BottleneckCSP-2模块
图 5  改进后的YOLOv5s网络
序号 M1 P1/104 Q1/ GLOPs M1 P2/104 Q2/ GLOPs (P1P2)/ 104 (Q1Q2)/ GLOPs
0 Focus 0.3520 0.3539 Conv 0.0640 0.0590 0.2880 0.2949
1 CBL 1.8560 0.4719 CBL 1.8560 0.4719 0.0000 0.0000
2 BottleneckCSP 1.9904 0.4981 BottleneckCSP-2 1.7792 0.4456 0.2112 0.0524
3 CBL 7.3984 0.4719 CBAM 0.0610 0.0156 ?0.0610 ?0.0156
4 BottleneckCSP 16.1152 1.0224 CBL 7.3984 0.4719 0.0000 0.0000
5 CBL 29.5424 0.4719 BottleneckCSP-2 10.7392 0.6816 5.3760 0.3408
6 BottleneckCSP 64.1792 1.0224 CBAM 0.2146 0.0137 ?0.2146 ?0.0137
7 SPP 65.6896 0.2621 CBL 29.5424 0.4719 0.0000 0.0000
8 BottleneckCSP 124.8768 0.4981 BottleneckCSP-2 60.8768 0.9699 3.3024 0.0524
9 CBL 118.0672 0.4719 CBAM 0.8290 0.0133 ?0.8290 ?0.0133
10 CBL 118.0672 0.4719 0.0000 0.0000
11 BottleneckCSP-2 111.7184 0.4456 13.1584 0.0524
12 CBAM 3.2866 0.0131 ?3.2866 ?0.0131
13 SPPF 65.6896 0.2621 0.0000 0.0000
总量 430.0672 5.5443 412.1224 4.8071 17.9448 0.7372
表 1  原YOLOv5s与YOLOv5s-PRO参数量及计算量的对比
图 6  YOLOv5s-PRO损失曲线
图 7  YOLOv5s-PRO的平均精度曲线
n Pr Re mAP-U mAP-H
103 0.9612 0.8552 0.9376 0.5240
105 0.9466 0.8656 0.9193 0.5245
104 0.9576 0.8830 0.9287 0.5233
106 0.9634 0.8824 0.9314 0.5224
100 0.9619 0.8763 0.9258 0.5216
88 0.9754 0.9000 0.9279 0.5195
118 0.9685 0.8590 0.9144 0.5206
109 0.9362 0.8603 0.9226 0.5157
82 0.9258 0.8850 0.9307 0.5133
97 0.9270 0.8835 0.9301 0.5127
108 0.9541 0.8679 0.9184 0.5132
114 0.9320 0.8758 0.9257 0.5121
119 0.9458 0.8516 0.9047 0.5144
92 0.9594 0.8445 0.9145 0.5129
107 0.9613 0.8485 0.9205 0.5118
98 0.9140 0.8568 0.9173 0.5112
112 0.9613 0.8254 0.9074 0.5118
96 0.9299 0.8733 0.9225 0.5100
87 0.9509 0.8831 0.9176 0.5104
94 0.9605 0.8530 0.9167 0.5097
表 2  YOLOv5s-PRO训练数据的统计表
图 8  验证集的测试结果
图 9  YOLOv5s深层火灾特征检测图样
图 10  YOLOv5s-PRO深层火灾特征检测图样
图 11  骨干网络中间特征层的分析
图 12  训练过程的损失曲线
图 13  烟火检测的验证结果
数据集 F1 F2 F1? F2 S1 S2 S1? S2 mAP1 mAP2 mAP1? mAP2 t1/ms t2/ms (t1? t2)/ms
95.28 96.74 ?1.46 90.81 92.63 ?1.82 93.12 94.69 ?1.57 6.53 6.09 0.44
90.33 91.61 ?1.28 83.38 89.78 ?6.40 87.41 90.70 ?3.29 6.51 6.11 0.40
88.78 90.89 ?2.11 81.61 87.52 ?5.91 84.42 89.21 ?4.79 6.57 6.17 0.40
均值 91.46 93.08 ?1.62 85.27 89.98 ?4.71 88.32 91.53 ?3.21 6.54 6.12 0.42
表 3  原YOLOv5s与YOLOv5s-PRO的测试性能
图 14  不同改进方案与YOLOv5s-PRO性能评价
图 15  重庆真武山隧道的火灾测试
图 16  重庆真武山隧道的火灾误检
类别 nd nn nf Pd/% Pa/%
火焰 381 11 7 97.11 98.16
烟雾 357 17 0 95.24 100.00
表 4  真武山隧道火灾视频测试结果的统计
1 胡嘉伟, 毛军, 刘斌, 等 隧道内横向偏置火源火灾烟气温度特性全尺寸试验研究[J]. 中国公路学报, 2020, 33 (11): 205- 214
HU Jia-wei, MAO Jun, LIU Bin, et al Full-scale experimental study on smoke temperature and fire characteristics occurred at deflected positions in tunnels[J]. China Journal of Highway and Transport, 2020, 33 (11): 205- 214
doi: 10.3969/j.issn.1001-7372.2020.11.019
2 王浩远, 梁煜, 张为 融合多分辨率表征的实时烟雾分割算法[J]. 浙江大学学报: 工学版, 2021, 55 (12): 2334- 2341
WANG Hao-yuan, LIANG Yu, ZHANG Wei Real-time smoke segmentation algorithm fused with multi-resolution representation[J]. Journal of Zhejiang University: Engineering Science, 2021, 55 (12): 2334- 2341
3 王琳, 姚新, 雷丹 公路隧道火灾初期视频火焰检测[J]. 中国公路学报, 2018, 31 (11): 121- 129
WANG Lin, YAO Xin, LEI Dan Video-based recognition of early fire flame in road tunnel[J]. China Journal of Highway and Transport, 2018, 31 (11): 121- 129
doi: 10.3969/j.issn.1001-7372.2018.11.013
4 梅建军, 张为 基于ViBe与机器学习的早期火灾检测算法[J]. 光学学报, 2018, 38 (7): 60- 67
MEI Jian-jun, ZHANG Wei Early fire detection algorithm based on ViBe and machine learning[J]. Acta Optica Sinica, 2018, 38 (7): 60- 67
5 WANG T, BU L P, YANG Z K, et al A new fire detection method using a multi-expert system based on color dispersion, similarity and centroid motion in indoor environment[J]. IEEE/CAA Journal of Automatica Sinica, 2020, 7 (1): 263- 275
6 肖潇, 孔凡芝, 刘金华 基于动静态特征的监控视频火灾检测算法[J]. 计算机科学, 2019, 46 (Supple.1): 284- 286
XIAO Xiao, KONG Fan-zhi, LIU Jin-hua Monitoring video fire detection algorithm based on dynamic characteristics and static characteristics[J]. Computer Science, 2019, 46 (Supple.1): 284- 286
7 LI Z L, MIHAYLOVA L S, ISUPOVA O, et al Autonomous flame detection in videos with a Dirichlet process Gaussian mixture color model[J]. IEEE Transactions on Industrial Informatics, 2018, 14 (3): 1146- 1154
doi: 10.1109/TII.2017.2768530
8 赵敏, 张为, 王鑫, 等 时空背景模型下结合多种纹理特征的烟雾检测[J]. 西安交通大学学报, 2018, 52 (8): 67- 73
ZHAO Min, ZHANG Wei, WANG Xin, et al A smoke detection algorithm with multi-texture feature exploration under a spatio temporal background model[J]. Journal of Xi'an Jiaotong University, 2018, 52 (8): 67- 73
doi: 10.7652/xjtuxb201808011
9 刘恺, 刘湘, 常丽萍, 等 基于YUV颜色空间和多特征融合的视频烟雾检测[J]. 传感技术学报, 2019, 32 (2): 237- 243
LIU Kai, LIU Xiang, CHANG Li-ping, et al Video smoke detection based on YUV color space and multiple feature fusion[J]. Chinese Journal of Sensors and Actuators, 2019, 32 (2): 237- 243
doi: 10.3969/j.issn.1004-1699.2019.02.014
10 李洪昌, 安明伟 基于总有界变分的森林火灾烟雾图像检测方法[J]. 电子测量与仪器学报, 2020, 34 (11): 211- 217
LI Hong-chang, AN Ming-wei Smoke image detection method of the forest fire based on total bounded variation[J]. Journal of Electronic Measurement and Instrumentation, 2020, 34 (11): 211- 217
doi: 10.13382/j.jemi.B2003042
11 程旭, 宋晨, 史金钢, 等 基于深度学习的通用目标检测研究综述[J]. 电子学报, 2021, 49 (7): 1428- 1438
CHENG Xu, SONG Chen, SHI Jin-gang, et al A survey of generic object detection methods based on deep learning[J]. Acta Electronica Sinica, 2021, 49 (7): 1428- 1438
doi: 10.12263/DZXB.20200570
12 HOU Y C, WANG H Q, WANG K Improved multi-scale flame detection method[J]. Chinese Journal of Liquid Crystals and Displays, 2021, 36 (5): 751- 759
doi: 10.37188/CJLCD.2020-0221
13 HU Y C, LU X B Real-time video fire smoke detection by utilizing spatial-temporal ConvNet features[J]. Multimedia Tools Appl, 2018, 77 (22): 29283- 29301
doi: 10.1007/s11042-018-5978-5
14 CHEN Z, ZHONG C, SHAO Y, et al Video fire recognition based on multi-channel convolutional neural network[J]. Journal of Physics: Conference Series, 2020, 1634 (1): 12020- 12027
doi: 10.1088/1742-6596/1634/1/012020
15 HOSSEINI A, HASHEMZADEH M, FARAJZADEH N UFS-Net: a unified flame and smoke detection method for early detection of fire in video surveillance applications using CNNs[J]. Journal of Computational Science, 2022, 61: 101638
doi: 10.1016/j.jocs.2022.101638
16 石磊, 张海刚, 杨金锋 基于改进型SSD的视频烟火检测算法[J]. 计算机应用与软件, 2021, 38 (12): 161- 167
SHI Lei, ZHANG Hai-gang, YANG Jin-feng Video-based fire and smoke detection based on improved SSD[J]. Computer Applications and Software, 2021, 38 (12): 161- 167
doi: 10.3969/j.issn.1000-386x.2021.12.027
17 赵媛媛, 朱军, 谢亚坤, 等 改进Yolo-v3的视频图像火焰实时检测算法[J]. 武汉大学学报: 信息科学版, 2021, 46 (3): 326- 334
ZHAO Yuan-yuan, ZHU Jun, XIE Ya-kun, et al A real-time video flame detection algorithm based on improved Yolo-v3[J]. Geomatics and Information Science of Wuhan University, 2021, 46 (3): 326- 334
18 缪伟志, 陆兆纳, 王俊龙, 等 基于视觉的火灾检测研究[J]. 森林工程, 2022, 38 (1): 86- 92
MIAO Wei-zhi, LU Zhao-na, WANG Jun-long, et al Fire detection research based on vision[J]. Forest Engineering, 2022, 38 (1): 86- 92
doi: 10.3969/j.issn.1006-8023.2022.01.011
19 YAN P C, SUN Q S, YIN N N, et al Detection of coal and gangue based on improved YOLOv5.1 which embedded scSE module[J]. Measurement, 2022, 188: 110530
doi: 10.1016/j.measurement.2021.110530
20 GE Z, LIU S, WANG F, et al. Yolox: exceeding yolo series in 2021 [EB/OL]. [2022-03-20]. https://arxiv.org/abs/2107.08430.
21 王莉, 何牧天, 徐硕, 等 基于YOLOv5s网络的垃圾分类和检测[J]. 包装工程, 2021, 42 (8): 50- 56
WANG Li, HE Mu-tian, XU Shuo, et al Garbage classification and detection based on YOLOv5s network[J]. Packaging Engineering, 2021, 42 (8): 50- 56
doi: 10.19554/j.cnki.1001-3563.2021.08.007
22 ZHU L L, GENG X, LI Z, et al Improving YOLOv5 with attention mechanism for detecting boulders from planetary images[J]. Remote Sensing, 2021, 13: 3776
doi: 10.3390/rs13183776
23 林森, 刘美怡, 陶志勇 采用注意力机制与改进YOLOv5的水下珍品检测[J]. 农业工程学报, 2021, 37 (18): 307- 314
LIN Sen, LIU Mei-yi, TAO Zhi-yong Detection of underwater treasures using attention mechanism and improved YOLOv5[J]. Transactions of the Chinese Society of Agricultural Engineering, 2021, 37 (18): 307- 314
doi: 10.11975/j.issn.1002-6819.2021.18.035
[1] 苏育挺,陆荣烜,张为. 基于注意力和自适应权重的车辆重识别算法[J]. 浙江大学学报(工学版), 2023, 57(4): 712-718.
[2] 曾耀,高法钦. 基于改进YOLOv5的电子元件表面缺陷检测算法[J]. 浙江大学学报(工学版), 2023, 57(3): 455-465.
[3] 兰欢,余建波. 基于深度学习三维成型的钢板表面缺陷检测[J]. 浙江大学学报(工学版), 2023, 57(3): 466-476.
[4] 曾菊香,王平辉,丁益东,兰林,蔡林熹,管晓宏. 面向节点分类的图神经网络节点嵌入增强模型[J]. 浙江大学学报(工学版), 2023, 57(2): 219-225.
[5] 鲁建厦,包秦,汤洪涛,邵益平,赵文彬. 无设备人体追踪系统的择优标签方法[J]. 浙江大学学报(工学版), 2023, 57(2): 415-425.
[6] 马骏驰,迪骁鑫,段宗涛,唐蕾. 程序表示学习综述[J]. 浙江大学学报(工学版), 2023, 57(1): 155-169.
[7] 叶晨,战洪飞,林颖俊,余军合,王瑞,钟武昌. 基于推理-情境感知激活模型的设计知识推荐[J]. 浙江大学学报(工学版), 2023, 57(1): 32-46.
[8] 刘近贞,陈飞,熊慧. 多尺度残差网络模型的开放式电阻抗成像算法[J]. 浙江大学学报(工学版), 2022, 56(9): 1789-1795.
[9] 王万良,王铁军,陈嘉诚,尤文波. 融合多尺度和多头注意力的医疗图像分割方法[J]. 浙江大学学报(工学版), 2022, 56(9): 1796-1805.
[10] 郝琨,王阔,王贝贝. 基于改进Mobilenet-YOLOv3的轻量级水下生物检测算法[J]. 浙江大学学报(工学版), 2022, 56(8): 1622-1632.
[11] 夏杰锋,唐武勤,杨强. 光伏航拍红外图像的热斑自动检测方法[J]. 浙江大学学报(工学版), 2022, 56(8): 1640-1647.
[12] 赵永胜,李瑞祥,牛娜娜,赵志勇. 数字孪生驱动的机身形状控制方法[J]. 浙江大学学报(工学版), 2022, 56(7): 1457-1463.
[13] 何立,庞善民. 结合年龄监督和人脸先验的语音-人脸图像重建[J]. 浙江大学学报(工学版), 2022, 56(5): 1006-1016.
[14] 张雪芹,李天任. 基于Cycle-GAN和改进DPN网络的乳腺癌病理图像分类[J]. 浙江大学学报(工学版), 2022, 56(4): 727-735.
[15] 褚晶辉,史李栋,井佩光,吕卫. 适用于目标检测的上下文感知知识蒸馏网络[J]. 浙江大学学报(工学版), 2022, 56(3): 503-509.