Please wait a minute...
浙江大学学报(工学版)
Journal of ZheJiang University (Engineering Science)  2022, Vol. 56 Issue (10): 1891-1899    DOI: 10.3785/j.issn.1008-973X.2022.10.001
    
Fire detection algorithm based on improved GhostNet-FCOS
Rong ZHANG(),Wei ZHANG*()
School of Microelectronics, Tianjin University, Tianjin 300072, China
Download: HTML     PDF(2209KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

A fire detection algorithm based on improved GhostNet-FCOS was proposed in view of the low detection accuracy and high complexity of existing fire detection algorithms. The algorithm was based on FCOS with reduced channel dimensions, and GhostNet was selected as the feature extraction network to implement a lightweight fire detection algorithm. Dynamic convolution was introduced to optimize the basic modules of the backbone without increasing width and depth, resulting in improved feature extraction ability for variable flames. A spatial attention module was introduced into the backbone network in order to optimize the expression of network spatial features. The definition of positive and negative samples and the regression loss function were improved to optimize the network’s attention to different areas in the ground truth box during the training process. The experimental results in self-built fire dataset and public dataset show that the algorithm has advantages in detection accuracy and model complexity. The detection accuracy of the algorithm in the self-built fire dataset was 90.9%, the amount of parameter was 4.58×106, and the floating point operation was 31.45×109.

Key words： fire detection      object detection      FCOS      GhostNet      dynamic convolution      attention module     
Received: 03 November 2021      Published: 25 October 2022
CLC:  TP 391  
Fund:  新一代人工智能科技重大专项资助项目(19ZXZNGX00030)
Corresponding Authors: Wei ZHANG     E-mail: rongzhang@tju.edu.cn;tjuzhangwei@tju.edu.cn
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Rong ZHANG
Wei ZHANG
Cite this article:

Rong ZHANG,Wei ZHANG. Fire detection algorithm based on improved GhostNet-FCOS. Journal of ZheJiang University (Engineering Science), 2022, 56(10): 1891-1899.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2022.10.001     OR     https://www.zjujournals.com/eng/Y2022/V56/I10/1891


基于改进GhostNet-FCOS的火灾检测算法

针对现有火灾检测算法检测精度不佳和算法模型复杂度过高的问题，提出基于改进GhostNet-FCOS的火灾检测算法. 该算法以目标检测网络FCOS为基础，降低通道维数并选用GhostNet作为特征提取网络，以实现轻量化火灾检测算法. 引入动态卷积，在不增加网络宽度和深度的情况下优化主干网络的基础模块，提高对形态多变的火焰图像的特征提取能力. 增加空间注意力模块，优化网络空间特征的表达. 改进正负样本定义和回归损失函数，优化训练过程中算法模型对标注框内不同区域的关注程度. 在自建火灾数据集和公开数据集中的实验结果表明，该算法在检测精度和模型复杂度方面具有优势. 该算法在自建火灾数据集中的检测精度为90.9%，参数量为4.58×106，浮点计算量为31.45×109.


关键词： 火灾检测,  目标检测,  FCOS,  GhostNet,  动态卷积,  注意力模块 
Fig.1 Overall architecture of fire detection algorithm based on improved GhostNet-FCOS
输入 操作 扩张系数 输出 注意力 步长
3 3×3 conv — 16 — 2
16 Ghost bottleneck 16 16 — 1
16 Ghost bottleneck 48 24 — 2
24 Ghost bottleneck 72 24 — 1
24 Ghost bottleneck 72 40 √ 2
40 Ghost bottleneck 120 40 √ 1
40 Ghost bottleneck 240 80 — 2
80 Ghost bottleneck 200 80 — 1
80 Ghost bottleneck 184 80 — 1
80 Ghost bottleneck 184 80 — 1
80 Ghost bottleneck 480 112 √ 1
112 Ghost bottleneck 672 112 √ 1
112 Ghost bottleneck 672 160 √ 2
160 Ghost bottleneck 960 160 — 1
160 Ghost bottleneck 960 160 √ 1
160 Ghost bottleneck 960 160 — 1
160 Ghost bottleneck 960 160 √ 1
160 1×1 conv — 960 — 1
Tab.1 Network structure of improved GhostNet
Fig.2 Ghost module combined with dynamic convolution
Fig.3 Spatial attention module
Fig.4 Central sampling strategy
Fig.5 Fire detection public dataset
检测网络-通道数 主干网络 AP50/% AR/% P/106 FLOPs/109 v/(帧·s?1)
FCOS-256 ResNet50 91.4 56.2 74.13 200.5 18.0
FCOS-128 ShuffleNetV2 1.0× (g=3) 83.9 51.7 3.06 31.99 33.5
FCOS-128 ShuffleNetV2 2.0× 85.6 53.9 7.34 41.68 32.3
FCOS-128 MobileNetV2 86.2 53.2 4.04 35.22 29.1
FCOS-128 MobileNetV3 86.6 52.5 4.74 33.35 29.6
FCOS-128 GhostNet 86.5 52.4 4.45 30.45 28.6
Tab.2 Comparison of lightweight backbones
算法 动态卷积模块 空间注意力模块 监督信号和损失函数 AP50/% AR/% P/106 v/(帧·s?1)
FCOS — — — 91.4 56.2 74.13 18.0
GhostNet-FCOS — — — 86.5 52.4 4.45 28.6
本文算法 √ — — 88.7 53.8 4.58 25.1
本文算法 √ √ — 90.1 54.2 4.58 24.1
本文算法 √ √ √ 90.9 55.1 4.58 24.1
Tab.3 Results of ablation study
Fig.6 Comparison of detection results before and after improving supervision signal and loss function
算法 TPR/% TNR/%
V1 V2 V3 V4 V5
文献[25] 92.50 94.21 — 95.45 —
文献[26] 92.25 94.59 — 96.15 —
文献[27] 93.00 94.98 — 96.50 —
文献[28] 93.10 95.70 — 100 —
文献[2] 96.15 100.0 94.29 100 95.20
文献[29] — 91.20 — 95.80 —
文献[3] 93.30 — — — 94.70
文献[30] 91.00 93.00 — 100 —
本文算法1 96.66 96.53 99.59 95.41 96.10
本文算法2 98.75 100 100 99.23 99.35
Tab.4 Comparison with fire detection algorithms based on traditional image features
Fig.7 Comparison of detection effect of Ghost-FCOS before and after improvement
算法 AP50/% AR/% P/106 FLOPs/109
YOLOV3 Tiny [31] 76.5 43.2 8.67 16.24
YOLOV3-MobileNet[32] 82.6 46.8 22.67 53.46
SSD-MobileNetV2 [33] 76.6 46.9 5.45 8.08
本文算法 90.9 55.1 4.58 31.45
Tab.5 Comparison with lightweight object detection network used in fire detection
Fig.8 Detection results of experimental fire and real fire
[1]   FOGGIA P, SAGGESE A, VENTO M Real-time fire detection for video-surveillance applications using a combination of experts based on color, shape, and motion[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2015, 25 (9): 1545- 1556
doi: 10.1109/TCSVT.2015.2392531
[2]   严云洋, 杜静, 高尚兵, 等 融合多特征的视频火焰检测[J]. 计算机辅助设计与图形学学报, 2015, 27 (3): 433- 440
YAN Yun-yang, DU Jing, GAO Shang-bing, et al Video flame detection based on fusion of multi-feature[J]. Journal of Computer-Aided Design and Computer Graphics, 2015, 27 (3): 433- 440
[3]   HASHEMZADEH M, ZADEMEHDI A Fire detection for video surveillance applications using ICA k-medoids-based color model and efficient spatio-temporal visual features[J]. Expert Systems with Applications, 2019, 130: 60- 78
doi: 10.1016/j.eswa.2019.04.019
[4]   MUHAMMAD K, AHMAD J, BAIK S W Early fire detection using convolutional neural networks during surveillance for effective disaster management[J]. Neurocomputing, 2018, 288: 30- 42
doi: 10.1016/j.neucom.2017.04.083
[5]   MUHAMMAD K, AHMAD J, MEHMOOD I, et al Convolutional neural networks based fire detection in surveillance videos[J]. IEEE Access, 2018, 6: 18174- 18183
doi: 10.1109/ACCESS.2018.2812835
[6]   MUHAMMAD K, AHMAD J, LV Z, et al Efficient deep CNN-based fire detection and localization in video surveillance applications[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2018, 49 (7): 1419- 1434
[7]   MUHAMMAD K, KHAN S, ELHOSENY M, et al Efficient fire detection for uncertain surveillance environment[J]. IEEE Transactions on Industrial Informatics, 2019, 15 (5): 3113- 3122
doi: 10.1109/TII.2019.2897594
[8]   LI S, YAN Q, LIU P An efficient fire detection method based on multiscale feature extraction, implicit deep supervision and channel attention mechanism[J]. IEEE Transactions on Image Processing, 2020, 29: 8467- 8475
doi: 10.1109/TIP.2020.3016431
[9]   JEON M, CHOI H S, LEE J, et al Multi-scale prediction for fire detection using convolutional neural network[J]. Fire Technology, 2021, 57 (5): 1- 19
[10]   CHAOXIA C Y, SHANG W W, ZHANG F Information-guided flame detection based on faster R-CNN[J]. IEEE Access, 2020, 8: 58923- 58932
doi: 10.1109/ACCESS.2020.2982994
[11]   晋耀, 张为 采用Anchor-Free网络结构的实时火灾检测算法[J]. 浙江大学学报: 工学版, 2020, 54 (12): 2430- 2436
JIN Yao, ZHANG Wei Real-time fire detection algorithm with anchor-free network architecture[J]. Journal of Zhejiang University: Engineering Science, 2020, 54 (12): 2430- 2436
[12]   李欣健, 张大胜, 孙利雷, 等 复杂场景下基于CNN的轻量火焰检测方法[J]. 模式识别与人工智能, 2021, 34 (5): 415- 422
LI Xin-jian, ZHANG Da-sheng, SUN Li-lei, et al CNN-based lightweight fire detection method in complex scenes[J]. Pattern Recognition and Artificial Intelligence, 2021, 34 (5): 415- 422
doi: 10.16451/j.cnki.issn1003-6059.202105004
[13]   KIM B, LEE J A video-based fire detection using deep learning models[J]. Applied Sciences, 2019, 9 (14): 2862
doi: 10.3390/app9142862
[14]   TIAN Z, SHEN C, CHEN H, et al. FCOS: fully convolutional one-stage object detection [C]// Proceedings of the IEEE International Conference on Computer Vision. Seoul: IEEE, 2019: 9627–9636.
[15]   HAN K, WANG Y, TIAN Q, et al. Ghostnet: more features from cheap operations [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 1580-1589.
[16]   LIN T Y, DOLLAR P, GIRSHICK R, et al. Feature pyramid networks for object detection [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 2117–2125.
[17]   YANG B, BENDER G, LE Q V, et al CondConv: conditionally parameterized convolutions for efficient inference[J]. Advances in Neural Information Processing Systems, 2019, 32: 1307- 1318
[18]   HU J, SHEN L, ALBANIE S, et al Squeeze-and-excitation networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42 (8): 2011- 2023
doi: 10.1109/TPAMI.2019.2913372
[19]   WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module [C]// European Conference on Computer Vision. Munich: Springer, 2018: 1-19.
[20]   REZATOFIGHI H, TSOI N, GWAK J, et al. Generalized intersection over union: a metric and a loss for bounding box regression [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 658-666.
[21]   LIN T Y, MAIRE M, BELONGIE S, et al. Microsoft coco: common objects in context [C]// European Conference on Computer Vision. Zurich: Springer, 2014: 740-755.
[22]   MA N, ZHANG X, ZHENG H T, et al. Shufflenet v2: practical guidelines for efficient CNN architecture design [C]// Proceedings of the European Conference on Computer Vision. Munich: Springer, 2018: 116-131.
[23]   SANDLER M, HOWARD A, ZHU M, et al. Mobilenetv2: inverted residuals and linear bottlenecks [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 4510-4520.
[24]   HOWARD A, SANDLER M, CHEN B, et al. Searching for mobilenetv3 [C]// International Conference on Computer Vision. Seoul: IEEE, 2019: 1314-1324.
[25]   KO B C, CHEONG K H, NAM J Y Fire detection based on vision sensor and support vector machines[J]. Fire Safety Journal, 2009, 44 (3): 322- 329
doi: 10.1016/j.firesaf.2008.07.006
[26]   BORGES P V K, IZQUIERDO E A probabilistic approach for vision-based fire detection in videos[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2010, 20 (5): 721- 731
doi: 10.1109/TCSVT.2010.2045813
[27]   TRUONG T X, KIM J M Fire flame detection in video sequences using multi-stage pattern recognition techniques[J]. Engineering Applications of Artificial Intelligence, 2012, 25 (7): 1365- 1372
doi: 10.1016/j.engappai.2012.05.007
[28]   KONG S G, JIN D, LI S Fast fire flame detection in surveillance video using logistic regression and temporal smoothing[J]. Fire Safety Journal, 2016, 79: 37- 43
doi: 10.1016/j.firesaf.2015.11.015
[29]   吴茜茵, 严云洋, 杜静, 等 多特征融合的火焰检测算法[J]. 智能系统学报, 2015, 10 (2): 240- 247
WU Xi-yin, YAN Yun-yang, DU Jing, et al Fire detection based on fusion of multiple features[J]. CAAI Transactions on Intelligent Systems, 2015, 10 (2): 240- 247
doi: 10.3969/j.issn.1673-4785.201406022
[30]   梅建军, 张为 基于ViBe与机器学习的早期火灾监测算法[J]. 光学学报, 2018, 38 (7): 60- 67
MEI Jian-jun, ZHANG Wei Early fire detection algorithm based on ViBe and machine learning[J]. Acta Optica Sinica, 2018, 38 (7): 60- 67
[31]   KANG L W, WANG I S, CHOU K L, et al. Image-based real-time fire detection using deep learning with data augmentation for vision-based surveillance applications [C]// 16th IEEE International Conference on Advanced Video and Signal Based Surveillance. Taipei: IEEE, 2019: 1-4.
[32]   DAI Z. Image flame detection method based on improved yolov3 [C]// IOP Conference Series: Earth and Environmental Science. Xi’an: IOP Publishing, 2021, 693(1): 012012.
[1] Na ZHANG,Xu-lei QI,Xiao-an BAO,Biao WU,Xiao-mei TU,Yu-ting JIN. Single-stage object detection algorithm based on optimizing position prediction[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(4): 783-794.
[2] Jing-hui CHU,Li-dong SHI,Pei-guang JING,Wei LV. Context-aware knowledge distillation network for object detection[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(3): 503-509.
[3] Kai DU,Guo-rong ZHU,Jiang-hua LU,Mu-ye PANG. Metal object detection method in wireless electric vehicle charging system[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(1): 56-62.
[4] Hao-yuan WANG,Yu LIANG,Wei ZHANG. Real-time smoke segmentation algorithm fused with multi-resolution representation[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(12): 2334-2341.
[5] Qing-qing LIU,Zhi-yong ZHOU,Guo-hua FAN,Xu-sheng QIAN,Ji-su HU,Guang-qiang CHEN,Ya-kang DAI. Semi-supervised learning segmentation method of liver CT images based on 3D scSE-UNet[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(11): 2033-2044.
[6] Ying-jie NIU,Yan-chen SU,Dun-cheng CHENG,Jia LIAO,Hai-bo ZHAO,Yong-qiang GAO. High-speed rail contact network U-holding nut fault detection algorithm[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(10): 1912-1921.
[7] Ying-jie XIA,Cong-yu OUYANG. Dynamic image background modeling method for detecting abandoned objects in highway[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(7): 1249-1255.
[8] Chen-bin ZHENG,Yong ZHANG,Hang HU,Ying-rui WU,Guang-jing HUANG. Object detection enhanced context model[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(3): 529-539.
[9] Yao JIN,Wei ZHANG. Real-time fire detection algorithm with Anchor-Free network architecture[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(12): 2430-2436.
[10] Yue DONG,Hua-jun FENG,Zhi-hai XU,Yue-ting CHEN,Qi LI. Attention Res-Unet: an efficient shadow detection algorithm[J]. Journal of ZheJiang University (Engineering Science), 2019, 53(2): 373-381.
[11] YE Fang-fang, XU Li. Real-time detection and discrimination of static objects and ghosts[J]. Journal of ZheJiang University (Engineering Science), 2015, 49(1): 181-185.
[12] XIE Di, TONG Ruo-feng, TANG Min, FENG Yang. Distinguishable method for video fire detection[J]. Journal of ZheJiang University (Engineering Science), 2012, 46(4): 698-704.
[13] XU Xue-mei, LI Li-xian, ZHANG Jian-yang, NI Lan, HUANG Zheng-yu, CAO Jian. Tracking algorithm of visible particles in transparent
liquid pharmaceutical[J]. Journal of ZheJiang University (Engineering Science), 2012, 46(10): 1822-1830.