Please wait a minute...
浙江大学学报(工学版)
浙江大学学报(工学版)  2021, Vol. 55 Issue (12): 2334-2341    DOI: 10.3785/j.issn.1008-973X.2021.12.013
计算机技术     
融合多分辨率表征的实时烟雾分割算法
王浩远(),梁煜,张为*()
天津大学 微电子学院，天津 300072
Real-time smoke segmentation algorithm fused with multi-resolution representation
Hao-yuan WANG(),Yu LIANG,Wei ZHANG*()
School of Microelectronics, Tianjin University, Tianjin 300072, China
 全文: PDF(1015 KB)   HTML
摘要：

针对烟雾分割领域缺乏应用于实际监控系统的实时烟雾分割算法的现况，提出高准确率的实时烟雾分割算法. 该算法利用轻量化的多分辨率卷积模块并行提取特征图，在获得丰富语义信息的同时满足实时分割的需求. 提出烟雾前景增强模块，使得烟雾像素点融合前景增强表征、避免背景信息干扰，分割准确率得以提高. 提出残差注意力模块，从通道、空间维度增强重要特征信息，抑制无效信息. 该算法在自建数据集上平均交并比为91.27%，每张图片预测时间为39.06 ms，网络权重为74.66 MB；在公开数据集上的对比结果表明，该算法综合检测性能优于其他烟雾检测算法. 该算法分割准确率高、检测速度快且模型轻量化，可以应用于实际视频监控系统.
关键词： 计算机视觉;  烟雾分割;  多分辨率模块;  烟雾前景增强模块;  残差注意力模块    
Abstract:

A high-accuracy real-time smoke segmentation algorithm was proposed, aiming at the lack of a real-time smoke segmentation algorithm applied to the actual monitoring systems in the field of smoke segmentation. A lightweight multi-resolution convolution module to extract feature maps in parallel was used in the algorithm, which met the needs of real-time segmentation while obtaining rich semantic information. A smoke foreground enhancement module was proposed to enable smoke pixels to be merged with their corresponding foreground enhancement representations, while avoiding the interference of background information, thereby improving the accuracy of segmentation. A residual attention module was proposed to enhance important feature information from the two dimensions of channel and space, and suppress invalid information. The algorithm had a mean intersection over union of 91.27% on the self-built data set, the prediction time of each picture was 39.06 ms, and the network weight was 74.66 MB. Comparison results on the public data set show that the comprehensive detection performance of this algorithm is better than that of other smoke detection algorithms. The algorithm has high segmentation accuracy, fast detection speed and the model is lightweight, which can be applied to actual video surveillance systems.
Key words: computer vision    smoke segmentation    multi-resolution module    smoke foreground enhancement module    residual attention module
收稿日期: 2021-01-15 出版日期: 2021-12-31
CLC:  TP 391.4  
基金资助: 国家重点研发计划课题（2020YFC1522405）；科技重大专项与工程（19ZXZNGX00030）；应急管理部消防救援局科研计划重点攻关项目（2019XFGG20）
通讯作者: 张为     E-mail: csxueqian@tju.edu.cn;tjuzhangwei@tju.edu.cn
作者简介: 王浩远(1996—)，男，硕士生，从事数字图像处理、模式识别研究. orcid.org/0000-0002-3002-4383. E-mail: csxueqian@tju.edu.cn
服务　　
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章　　
王浩远
梁煜
张为

引用本文:

王浩远,梁煜,张为. 融合多分辨率表征的实时烟雾分割算法[J]. 浙江大学学报(工学版), 2021, 55(12): 2334-2341.

Hao-yuan WANG,Yu LIANG,Wei ZHANG. Real-time smoke segmentation algorithm fused with multi-resolution representation. Journal of ZheJiang University (Engineering Science), 2021, 55(12): 2334-2341.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2021.12.013    或    https://www.zjujournals.com/eng/CN/Y2021/V55/I12/2334

图 1  融合多分辨率表征的主干网络架构
阶段 模块数 分支数 各分支卷积单元数 各分支输出尺寸
1 1 1 $\left[ 2 \right]$ $ \left[ {224 \times 224 \times 64} \right] $
2 1 2 $\left[ \begin{gathered} 2 \hfill \\ 3 \hfill \\ \end{gathered} \right]$ $\left[ \begin{gathered} 224 \times 224 \times 16 \\ 112 \times 112 \times 36 \\ \end{gathered} \right]$
3 4 3 $\left[ \begin{gathered} 2 \hfill \\ 3 \hfill \\ 4 \hfill \\ \end{gathered} \right]$ $\left[ \begin{gathered} 224 \times 224 \times 16 \\ 112 \times 112 \times 36 \\ 56 \times 56 \times 72 \\ \end{gathered} \right]$
4 3 4 $\left[ \begin{gathered} 2 \hfill \\ 3 \hfill \\ 4 \hfill \\ 4 \hfill \\ \end{gathered} \right]$ $\left[ \begin{gathered} 224 \times 224 \times 16 \\ 112 \times 112 \times 36 \\ 56 \times 56 \times 72 \\ 28 \times 28 \times 144 \\ \end{gathered} \right]$
表 1  主干网络参数设置
图 2  烟雾前景增强模块
图 3  残差注意力模块
图 4  实验室的拍摄视频数据
图 5  经典语义分割网络与本研究算法定性比较
算法 mIoU/% T/ms P/MB
FCN 88.91 60.61 434.11
PSPNet 89.86 56.82 385.13
DeepLabV3 90.92 86.21 534.14
DeepLabV3+ 91.95 63.69 344.12
本研究 91.27 39.06 74.66
表 2  经典语义分割网络与本研究算法的分割结果对比
视频名称 视频描述 视频帧数/帧
sBehindtheFence 距离远、场景复杂 630
sBtFence 距离远、场景复杂 1400
sMoky 烟雾稀薄、快速运动 900
sWasteBasket 室内、有干扰（白墙） 900
sWindow 室外、运动缓慢 244
表 3  公开数据集描述
图 6  公开数据集部分检测结果
视频名称 本研究算法 文献[20] 文献[19] 文献[18]
RTPR RTNR RTPR RTNR RTPR RTNR RTPR RTNR
sBehindtheFence 98.64 100.00 98.26 94.60 97.20 96.27 94.72 100.00
sBtFence 98.81 100.00 98.20 100.00 98.17 100.00 99.08 100.00
sMoky 98.27 100.00 98.85 100.00 99.68 100.00 86.23 100.00
sWasteBasket 99.50 96.84 99.41 100.00 97.18 98.36 99.89 92.60
sWindow 98.46 97.87 98.40 100.00 98.10 100.00 94.30 100.00
表 4  现有烟雾检测算法与本研究算法的检测结果对比
算法 烟雾前景增强模块 残差注意力模块 mIoU/% T/ms P/MB
文献[8] √ √ 91.84 47.63 93.24
Ours1 ? ? 89.45 34.80 60.00
Ours2 √ ? 90.83 38.52 73.24
本研究 √ √ 91.27 39.06 74.66
表 5  消融实验结果
图 7  消融实验结果定性比较
1 TAO C, ZHANG J, WANG P. Smoke detection based on deep convolutional neural networks [C]// 2016 International Conference on Industrial Informatics-Computing Technology, Intelligent Technology, Industrial Information Integration. Wuhan: IEEE, 2016: 150-153.
2 SHRIVASTAVA M, MATLANI P. A smoke detection algorithm based on K-means segmentation [C]// 2016 International Conference on Audio, Language and Image Processing. Shanghai: IEEE, 2016: 301-305.
3 FILONENKO A, HERNÁNDEZ D C, JO K H Fast smoke detection for video surveillance using CUDA[J]. IEEE Transactions on Industrial Informatics, 2017, 14 (2): 725- 733
4 LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 3431-3440.
5 YUAN F, ZHANG L, XIA X, et al Deep smoke segmentation[J]. Neurocomputing, 2019, 357: 248- 260
doi: 10.1016/j.neucom.2019.05.011
6 XU G, ZHANG Y, ZHANG Q, et al Video smoke detection based on deep saliency network[J]. Fire Safety Journal, 2019, 105: 277- 285
doi: 10.1016/j.firesaf.2019.03.004
7 HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
8 SUN K, XIAO B, LIU D, et al. Deep high-resolution representation learning for human pose estimation [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 5693-5703.
9 HOWARD A G, ZHU M, CHEN B, et al. MobileNets: efficient convolutional neural networks for mobile vision applications [EB/OL]. [2020-11-16]. https://arxiv.org/pdf/1704.04861.pdf.
10 ZHAO H, SHI J, QI X, et al. Pyramid scene parsing network [C]// Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 2881-2890.
11 CHEN L C, PAPANDREOU G, SCHROFF F, et al. Rethinking atrous convolution for semantic image segmentation [EB/OL]. [2020-12-10]. https://arxiv.org/pdf/1706.05587.pdf.
12 CHEN L C, PAPANDREOU G, KOKKINOS I, et al DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40 (4): 834- 848
doi: 10.1109/TPAMI.2017.2699184
13 HE K, ZHANG X, REN S, et al Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37 (9): 1904- 1916
doi: 10.1109/TPAMI.2015.2389824
14 YUAN Y, CHEN X, WANG J, et. Object-contextual representations for semantic segmentation[C]// Computer Vision-ECCV 2020. [S.l.]: Springer, 2020: 173-190.
15 WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module [C]// Proceedings of the European Conference on Computer Vision. Munich: Springer, 2018: 3-19.
16 HU J, SHEN L, SUN G. Squeeze-and-excitation networks [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 7132-7141.
17 CHEN L C, ZHU Y, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation [C]// Proceedings of the European Conference on Computer Vision. Munich: Springer, 2018: 801-818.
18 BESBES O, BENAZZA-BENYAHIA A. A Novel video-based smoke detection method based on color invariants [C]// 2016 IEEE International Conference on Acoustics, Speech and Signal Processing. Shanghai: IEEE, 2016: 1911-1915.
19 赵敏, 张为, 王鑫, 等 时空背景模型下结合多种纹理特征的烟雾检测[J]. 西安交通大学学报, 2018, 52 (8): 67- 73
ZHAO Min, ZHANG Wei, Wang Xin, et al A smoke detection algorithm with multi-texture feature exploration under a spatio-temporal background model[J]. Journal of Xi’an Jiaotong University, 2018, 52 (8): 67- 73
20 汪梓艺, 苏育挺, 刘艳艳, 等 一种改进DeeplabV3网络的烟雾分割算法[J]. 西安电子科技大学学报, 2019, 46 (6): 52- 59
WANG Zi-yi, SU Yu-ting, LIU Yan-yan, et al Algorithm for segmentation of smoke using the improved DeeplabV3 network[J]. Journal of Xidian University, 2019, 46 (6): 52- 59
[1] 晋耀,张为. 采用Anchor-Free网络结构的实时火灾检测算法[J]. 浙江大学学报(工学版), 2020, 54(12): 2430-2436.
[2] 潘翔 马德强 吴贻军 张光富 姜哲圣. 基于视觉着陆的无人机俯仰角与高度估计[J]. , 2009, 43(4): 692-696.
[3] 漆随平 张宏建 骆志坚 周洪亮. 虚拟多传感器信息融合的在线钢材视觉检测[J]. J4, 2005, 39(9): 1363-1367.