Please wait a minute...
Journal of ZheJiang University (Engineering Science)  2023, Vol. 57 Issue (4): 784-794    DOI: 10.3785/j.issn.1008-973X.2023.04.016
    
Improved YOLOv5s flame and smoke detection method in road tunnels
Qing-lu MA1,2(),Jia-ping LU1,Xiao-yao TANG1,Xue-feng DUAN3
1. School of Traffic and Transportation, Chongqing Jiaotong University, Chongqing 400074, China
2. Chongqing Key Laboratory of "Human-Vehicle-Road" Cooperation and Safety for Mountain Complex Environment, Chongqing 400074, China
3. Ningxia Jiaotou Expressway Management Limited Company, Yinchuan 750000, China
Download: HTML     PDF(3978KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

An improved YOLOv5s for visual detection of smoke and fire in early-stage road tunnel fires was proposed to solve the problem of smoke and fire confusion and the requirement for real-time detection. The convolutional block attention module (CBAM) was introduced into YOLOv5s to improve the accuracy of detecting smoke with obscure contour features and initial tunnel flame with crucial features. The Focus module in the backbone network was replaced, the number of convolutional layers in BottleneckCSP was reduced, and the efficiency of the smoke and flame feature extraction network was improved. The CIoU was used to replace the original GIoU loss function to accelerate the convergence rate of the model. A data set containing 10 000 images of tunnel smoke and flame was used as the training sample. YOLOv5s and improved YOLOv5s-PRO were used for comparative test analysis. The model was validated by using the video data of the Zhenwu Mountain tunnel fire that occurred on March 6, 2021, in Chongqing, China. The experimental results showed that the detection accuracy of the algorithm reached up to 91.53%, which was 3.21% higher than YOLOv5s, and the detection speed reached 6.12 ms, which was 0.42 ms better than YOLOv5s. The YOLOv5s-PRO has higher detection accuracy and a faster rate, which can be applied to smoke and flame detection of actual road tunnel.



Key wordstunnel engineering      tunnel flame and smoke detection      attention module      deep learning      YOLOv5s     
Received: 03 April 2022      Published: 21 April 2023
CLC:  U 459  
Fund:  国家社会科学基金资助项目(20VYJ023);宁夏回族自治区交通运输厅科技资助项目(NJGF20200301)
Cite this article:

Qing-lu MA,Jia-ping LU,Xiao-yao TANG,Xue-feng DUAN. Improved YOLOv5s flame and smoke detection method in road tunnels. Journal of ZheJiang University (Engineering Science), 2023, 57(4): 784-794.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2023.04.016     OR     https://www.zjujournals.com/eng/Y2023/V57/I4/784


改进YOLOv5s的公路隧道烟火检测方法

针对公路隧道初期火灾烟火混淆且检测实时性要求高的问题,提出改进YOLOv5s的隧道烟火视觉检测方法. 该方法通过在YOLOv5s中引入卷积注意力模块(CBAM),提高对轮廓特征不明显的隧道烟雾及初期火焰重要特征检测的准确率. 替换骨干网络中的Focus模块,降低BottleneckCSP的卷积层数目,提升烟火特征提取网络效率. 用CIoU替换原有的GIoU损失函数,加快模型的收敛速度. 实验以10 000张隧道烟火数据集为训练样本,用YOLOv5s和改进后的YOLOv5s-PRO进行对比试验分析,用2021年3月6日重庆真武山隧道火灾视频数据验证模型. 实验结果表明,该算法的检测精度达到91.53%,比YOLOv5s提高了3.21%,检测速度达到6.12 ms,比YOLOv5s提高了0.42 ms,检测精度较高,速度较快,可以应用于实际公路隧道的烟火检测.


关键词: 隧道工程,  隧道烟火检测,  注意力模块,  深度学习,  YOLOv5s 
Fig.1 YOLOv5s frame structure
Fig.2 Structure diagram of CBAM frame
Fig.3 Schematic diagram of replacement structure
Fig.4 BottleneckCSP-2 modular
Fig.5 Improved YOLOv5s network
序号 M1 P1/104 Q1/ GLOPs M1 P2/104 Q2/ GLOPs (P1P2)/ 104 (Q1Q2)/ GLOPs
0 Focus 0.3520 0.3539 Conv 0.0640 0.0590 0.2880 0.2949
1 CBL 1.8560 0.4719 CBL 1.8560 0.4719 0.0000 0.0000
2 BottleneckCSP 1.9904 0.4981 BottleneckCSP-2 1.7792 0.4456 0.2112 0.0524
3 CBL 7.3984 0.4719 CBAM 0.0610 0.0156 ?0.0610 ?0.0156
4 BottleneckCSP 16.1152 1.0224 CBL 7.3984 0.4719 0.0000 0.0000
5 CBL 29.5424 0.4719 BottleneckCSP-2 10.7392 0.6816 5.3760 0.3408
6 BottleneckCSP 64.1792 1.0224 CBAM 0.2146 0.0137 ?0.2146 ?0.0137
7 SPP 65.6896 0.2621 CBL 29.5424 0.4719 0.0000 0.0000
8 BottleneckCSP 124.8768 0.4981 BottleneckCSP-2 60.8768 0.9699 3.3024 0.0524
9 CBL 118.0672 0.4719 CBAM 0.8290 0.0133 ?0.8290 ?0.0133
10 CBL 118.0672 0.4719 0.0000 0.0000
11 BottleneckCSP-2 111.7184 0.4456 13.1584 0.0524
12 CBAM 3.2866 0.0131 ?3.2866 ?0.0131
13 SPPF 65.6896 0.2621 0.0000 0.0000
总量 430.0672 5.5443 412.1224 4.8071 17.9448 0.7372
Tab.1 Comparison of parameter quantity and calculation quantity of original YOLOv5s and YOLOv5s-PRO structure
Fig.6 YOLOv5s-PRO loss curve
Fig.7 Average accuracy curve of YOLOv5s-PRO
n Pr Re mAP-U mAP-H
103 0.9612 0.8552 0.9376 0.5240
105 0.9466 0.8656 0.9193 0.5245
104 0.9576 0.8830 0.9287 0.5233
106 0.9634 0.8824 0.9314 0.5224
100 0.9619 0.8763 0.9258 0.5216
88 0.9754 0.9000 0.9279 0.5195
118 0.9685 0.8590 0.9144 0.5206
109 0.9362 0.8603 0.9226 0.5157
82 0.9258 0.8850 0.9307 0.5133
97 0.9270 0.8835 0.9301 0.5127
108 0.9541 0.8679 0.9184 0.5132
114 0.9320 0.8758 0.9257 0.5121
119 0.9458 0.8516 0.9047 0.5144
92 0.9594 0.8445 0.9145 0.5129
107 0.9613 0.8485 0.9205 0.5118
98 0.9140 0.8568 0.9173 0.5112
112 0.9613 0.8254 0.9074 0.5118
96 0.9299 0.8733 0.9225 0.5100
87 0.9509 0.8831 0.9176 0.5104
94 0.9605 0.8530 0.9167 0.5097
Tab.2 Statistical table of YOLOv5s-PRO training data
Fig.8 Test results of verification set
Fig.9 YOLOv5s deep fire feature detection drawing
Fig.10 YOLOv5s-PRO deep fire feature detection drawing
Fig.11 Analysis of middle characteristic layer of backbone network
Fig.12 Loss curve during training
Fig.13 Verification results of smoke and flame detection
数据集 F1 F2 F1? F2 S1 S2 S1? S2 mAP1 mAP2 mAP1? mAP2 t1/ms t2/ms (t1? t2)/ms
95.28 96.74 ?1.46 90.81 92.63 ?1.82 93.12 94.69 ?1.57 6.53 6.09 0.44
90.33 91.61 ?1.28 83.38 89.78 ?6.40 87.41 90.70 ?3.29 6.51 6.11 0.40
88.78 90.89 ?2.11 81.61 87.52 ?5.91 84.42 89.21 ?4.79 6.57 6.17 0.40
均值 91.46 93.08 ?1.62 85.27 89.98 ?4.71 88.32 91.53 ?3.21 6.54 6.12 0.42
Tab.3 Test performance of original YOLOv5s and YOLOv5s-PRO structure %
Fig.14 Performance evaluation of improved schemes and YOLOv5s-PRO
Fig.15 Fire test of Zhenwu Mountain Tunnel in Chongqing
Fig.16 Fire misdiagnosis of Zhenwu Mountain Tunnel in Chongqing
类别 nd nn nf Pd/% Pa/%
火焰 381 11 7 97.11 98.16
烟雾 357 17 0 95.24 100.00
Tab.4 Statistics of Zhenwu Mountain Tunnel fire video test results
[1]   胡嘉伟, 毛军, 刘斌, 等 隧道内横向偏置火源火灾烟气温度特性全尺寸试验研究[J]. 中国公路学报, 2020, 33 (11): 205- 214
HU Jia-wei, MAO Jun, LIU Bin, et al Full-scale experimental study on smoke temperature and fire characteristics occurred at deflected positions in tunnels[J]. China Journal of Highway and Transport, 2020, 33 (11): 205- 214
doi: 10.3969/j.issn.1001-7372.2020.11.019
[2]   王浩远, 梁煜, 张为 融合多分辨率表征的实时烟雾分割算法[J]. 浙江大学学报: 工学版, 2021, 55 (12): 2334- 2341
WANG Hao-yuan, LIANG Yu, ZHANG Wei Real-time smoke segmentation algorithm fused with multi-resolution representation[J]. Journal of Zhejiang University: Engineering Science, 2021, 55 (12): 2334- 2341
[3]   王琳, 姚新, 雷丹 公路隧道火灾初期视频火焰检测[J]. 中国公路学报, 2018, 31 (11): 121- 129
WANG Lin, YAO Xin, LEI Dan Video-based recognition of early fire flame in road tunnel[J]. China Journal of Highway and Transport, 2018, 31 (11): 121- 129
doi: 10.3969/j.issn.1001-7372.2018.11.013
[4]   梅建军, 张为 基于ViBe与机器学习的早期火灾检测算法[J]. 光学学报, 2018, 38 (7): 60- 67
MEI Jian-jun, ZHANG Wei Early fire detection algorithm based on ViBe and machine learning[J]. Acta Optica Sinica, 2018, 38 (7): 60- 67
[5]   WANG T, BU L P, YANG Z K, et al A new fire detection method using a multi-expert system based on color dispersion, similarity and centroid motion in indoor environment[J]. IEEE/CAA Journal of Automatica Sinica, 2020, 7 (1): 263- 275
[6]   肖潇, 孔凡芝, 刘金华 基于动静态特征的监控视频火灾检测算法[J]. 计算机科学, 2019, 46 (Supple.1): 284- 286
XIAO Xiao, KONG Fan-zhi, LIU Jin-hua Monitoring video fire detection algorithm based on dynamic characteristics and static characteristics[J]. Computer Science, 2019, 46 (Supple.1): 284- 286
[7]   LI Z L, MIHAYLOVA L S, ISUPOVA O, et al Autonomous flame detection in videos with a Dirichlet process Gaussian mixture color model[J]. IEEE Transactions on Industrial Informatics, 2018, 14 (3): 1146- 1154
doi: 10.1109/TII.2017.2768530
[8]   赵敏, 张为, 王鑫, 等 时空背景模型下结合多种纹理特征的烟雾检测[J]. 西安交通大学学报, 2018, 52 (8): 67- 73
ZHAO Min, ZHANG Wei, WANG Xin, et al A smoke detection algorithm with multi-texture feature exploration under a spatio temporal background model[J]. Journal of Xi'an Jiaotong University, 2018, 52 (8): 67- 73
doi: 10.7652/xjtuxb201808011
[9]   刘恺, 刘湘, 常丽萍, 等 基于YUV颜色空间和多特征融合的视频烟雾检测[J]. 传感技术学报, 2019, 32 (2): 237- 243
LIU Kai, LIU Xiang, CHANG Li-ping, et al Video smoke detection based on YUV color space and multiple feature fusion[J]. Chinese Journal of Sensors and Actuators, 2019, 32 (2): 237- 243
doi: 10.3969/j.issn.1004-1699.2019.02.014
[10]   李洪昌, 安明伟 基于总有界变分的森林火灾烟雾图像检测方法[J]. 电子测量与仪器学报, 2020, 34 (11): 211- 217
LI Hong-chang, AN Ming-wei Smoke image detection method of the forest fire based on total bounded variation[J]. Journal of Electronic Measurement and Instrumentation, 2020, 34 (11): 211- 217
doi: 10.13382/j.jemi.B2003042
[11]   程旭, 宋晨, 史金钢, 等 基于深度学习的通用目标检测研究综述[J]. 电子学报, 2021, 49 (7): 1428- 1438
CHENG Xu, SONG Chen, SHI Jin-gang, et al A survey of generic object detection methods based on deep learning[J]. Acta Electronica Sinica, 2021, 49 (7): 1428- 1438
doi: 10.12263/DZXB.20200570
[12]   HOU Y C, WANG H Q, WANG K Improved multi-scale flame detection method[J]. Chinese Journal of Liquid Crystals and Displays, 2021, 36 (5): 751- 759
doi: 10.37188/CJLCD.2020-0221
[13]   HU Y C, LU X B Real-time video fire smoke detection by utilizing spatial-temporal ConvNet features[J]. Multimedia Tools Appl, 2018, 77 (22): 29283- 29301
doi: 10.1007/s11042-018-5978-5
[14]   CHEN Z, ZHONG C, SHAO Y, et al Video fire recognition based on multi-channel convolutional neural network[J]. Journal of Physics: Conference Series, 2020, 1634 (1): 12020- 12027
doi: 10.1088/1742-6596/1634/1/012020
[15]   HOSSEINI A, HASHEMZADEH M, FARAJZADEH N UFS-Net: a unified flame and smoke detection method for early detection of fire in video surveillance applications using CNNs[J]. Journal of Computational Science, 2022, 61: 101638
doi: 10.1016/j.jocs.2022.101638
[16]   石磊, 张海刚, 杨金锋 基于改进型SSD的视频烟火检测算法[J]. 计算机应用与软件, 2021, 38 (12): 161- 167
SHI Lei, ZHANG Hai-gang, YANG Jin-feng Video-based fire and smoke detection based on improved SSD[J]. Computer Applications and Software, 2021, 38 (12): 161- 167
doi: 10.3969/j.issn.1000-386x.2021.12.027
[17]   赵媛媛, 朱军, 谢亚坤, 等 改进Yolo-v3的视频图像火焰实时检测算法[J]. 武汉大学学报: 信息科学版, 2021, 46 (3): 326- 334
ZHAO Yuan-yuan, ZHU Jun, XIE Ya-kun, et al A real-time video flame detection algorithm based on improved Yolo-v3[J]. Geomatics and Information Science of Wuhan University, 2021, 46 (3): 326- 334
[18]   缪伟志, 陆兆纳, 王俊龙, 等 基于视觉的火灾检测研究[J]. 森林工程, 2022, 38 (1): 86- 92
MIAO Wei-zhi, LU Zhao-na, WANG Jun-long, et al Fire detection research based on vision[J]. Forest Engineering, 2022, 38 (1): 86- 92
doi: 10.3969/j.issn.1006-8023.2022.01.011
[19]   YAN P C, SUN Q S, YIN N N, et al Detection of coal and gangue based on improved YOLOv5.1 which embedded scSE module[J]. Measurement, 2022, 188: 110530
doi: 10.1016/j.measurement.2021.110530
[20]   GE Z, LIU S, WANG F, et al. Yolox: exceeding yolo series in 2021 [EB/OL]. [2022-03-20]. https://arxiv.org/abs/2107.08430.
[21]   王莉, 何牧天, 徐硕, 等 基于YOLOv5s网络的垃圾分类和检测[J]. 包装工程, 2021, 42 (8): 50- 56
WANG Li, HE Mu-tian, XU Shuo, et al Garbage classification and detection based on YOLOv5s network[J]. Packaging Engineering, 2021, 42 (8): 50- 56
doi: 10.19554/j.cnki.1001-3563.2021.08.007
[22]   ZHU L L, GENG X, LI Z, et al Improving YOLOv5 with attention mechanism for detecting boulders from planetary images[J]. Remote Sensing, 2021, 13: 3776
doi: 10.3390/rs13183776
[23]   林森, 刘美怡, 陶志勇 采用注意力机制与改进YOLOv5的水下珍品检测[J]. 农业工程学报, 2021, 37 (18): 307- 314
LIN Sen, LIU Mei-yi, TAO Zhi-yong Detection of underwater treasures using attention mechanism and improved YOLOv5[J]. Transactions of the Chinese Society of Agricultural Engineering, 2021, 37 (18): 307- 314
doi: 10.11975/j.issn.1002-6819.2021.18.035
[1] Yu-ting SU,Rong-xuan LU,Wei ZHANG. Vehicle re-identification algorithm based on attention mechanism and adaptive weight[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(4): 712-718.
[2] Yao ZENG,Fa-qin GAO. Surface defect detection algorithm of electronic components based on improved YOLOv5[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(3): 455-465.
[3] Huan LAN,Jian-bo YU. Steel surface defect detection based on deep learning 3D reconstruction[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(3): 466-476.
[4] Ju-xiang ZENG,Ping-hui WANG,Yi-dong DING,Lin LAN,Lin-xi CAI,Xiao-hong GUAN. Graph neural network based node embedding enhancement model for node classification[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(2): 219-225.
[5] Jian-sha LU,Qin BAO,Hong-tao TANG,Yi-ping SHAO,Wen-bin ZHAO. Optimal tag selection method for device-free human tracking system[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(2): 415-425.
[6] Jun-chi MA,Xiao-xin DI,Zong-tao DUAN,Lei TANG. Survey on program representation learning[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(1): 155-169.
[7] Chen YE,Hong-fei ZHAN,Ying-jun LIN,Jun-he YU,Rui WANG,Wu-chang ZHONG. Design knowledge recommendation based on inference-context-aware activation model[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(1): 32-46.
[8] Jin-zhen LIU,Fei CHEN,Hui XIONG. Open electrical impedance imaging algorithm based on multi-scale residual network model[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(9): 1789-1795.
[9] Wan-liang WANG,Tie-jun WANG,Jia-cheng CHEN,Wen-bo YOU. Medical image segmentation method combining multi-scale and multi-head attention[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(9): 1796-1805.
[10] Kun HAO,Kuo WANG,Bei-bei WANG. Lightweight underwater biological detection algorithm based on improved Mobilenet-YOLOv3[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(8): 1622-1632.
[11] Yong-sheng ZHAO,Rui-xiang LI,Na-na NIU,Zhi-yong ZHAO. Shape control method of fuselage driven by digital twin[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(7): 1457-1463.
[12] Li HE,Shan-min PANG. Face reconstruction from voice based on age-supervised learning and face prior information[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(5): 1006-1016.
[13] Xue-qin ZHANG,Tian-ren LI. Breast cancer pathological image classification based on Cycle-GAN and improved DPN network[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(4): 727-735.
[14] Jing-hui CHU,Li-dong SHI,Pei-guang JING,Wei LV. Context-aware knowledge distillation network for object detection[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(3): 503-509.
[15] Ruo-ran CHENG,Xiao-li ZHAO,Hao-jun ZHOU,Han-chen YE. Review of Chinese font style transfer research based on deep learning[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(3): 510-519, 530.