Please wait a minute...
浙江大学学报(工学版)  2022, Vol. 56 Issue (11): 2156-2167    DOI: 10.3785/j.issn.1008-973X.2022.11.006
机械与能源工程     
基于混合域注意力YOLOv4的输送带纵向撕裂多维度检测
李飞1,2(),胡坤1,2,3,*(),张勇2,王文善1,蒋浩1
1. 安徽理工大学 机械工程学院,安徽 淮南 232001
2. 深部煤矿采动响应与灾害防控国家重点实验室,安徽 淮南 232001
3. 安徽理工大学 环境友好材料与职业健康研究院,安徽 芜湖 241003
Multi-dimensional detection of longitudinal tearing of conveyor belt based on YOLOv4 of hybrid domain attention
Fei LI1,2(),Kun HU1,2,3,*(),Yong ZHANG2,Wen-shan WANG1,Hao JIANG1
1. School of Mechanical Engineering, Anhui University of Science and Technology, Huainan 232001, China
2. State Key Laboratory of Mining Response and Disaster Prevention and Control in Deep Coal Mines, Anhui University of Science and Technology, Huainan 232001, China
3. Institute of Environment-friendly Materials and Occupational Health, Anhui University of Science and Technology, Wuhu, 241003, China
 全文: PDF(8267 KB)   HTML
摘要:

针对输送带纵向撕裂目标检测维度单一、模型复杂度高等问题,提出一种高效的MobileNetv3及YOLOv4集成网络输送带纵向撕裂多维度实时检测方法. 基于YOLOv4目标识别算法,通过将轻量化网络MobileNetv3代替CSPDarknet53作为骨干网络,结合高效通道域ECA模块和空间域注意力机制(STNet)构建混合域注意力网络(ECSNet),改进了MobileNetv3嵌入ECSNet,并且提升了模型对空间和通道的关注度. 引入深度可分离卷积块代替网络中3*3卷积,并将YOLOv4的检测头(Prediction Heads)缩减为2种尺度,轻量化模型降低网络复杂度和训练难度,完成ECSMv3_YOLOv4模型的搭建,使用K-means聚类6个Anchors预测目标框高宽,提高网络对表面撕裂的检测性能. 研制带式输送机多维度智能巡检样机,采集制作输送带多维度面的纵向撕裂数据集,开展网络模型的训练、测试、识别和定位实验. 结果表明,提出算法在测试集中的平均识别准确率为97.8%,识别速度为37 帧/s,模型的计算量和参数量为4.882 G和8.851 M,通过试验不同的网络模型效果和改变光照强度,该方法体现出检测精度高、速度快和轻量化等优点,具备更强的适应性和抗干扰能力.

关键词: 纵向撕裂多维度检测MobileNetv3混合域注意力机制YOLOv4轻量化    
Abstract:

An efficient MobileNetv3 and YOLOv4 integrated network multi-dimensional real-time detection method for longitudinal tearing of conveyor belt was proposed to aim at the problem of single dimension and high complexity of model in the detection of the longitudinal tearing target of the conveyor belt. The lightweight network MobileNetv3 based on the object detection algorithm of YOLOv4 was used to replace CSPDarknet53 as the backbone network of YOLOv4. The ECSNet was constructed by combining efficient channel domain ECA model and spatial transformer network (STNet). The ECSNet was embedded in MobileNetv3 to improve the attention of model to space and channels. The deep separable convolution block was introduced to replace the 3*3 convolution in the network and the Prediction Heads of YOLOv4 were reduced to two scales. The network model was lightened, the complexity and training difficulty were reduced and ECSMv3_YOLOv4 model was built. The K-means was used to cluster six Anchors to predict the height and width of the bounding box, which improved the detection performance of the network for surface tearing. The multi-dimensional intelligent inspection prototype of belt conveyor was developed, the longitudinal tear data set of multi-dimensional surface of conveyor belt was collected and made. The training, testing, identification and positioning experiments of network model were carried out. The results show that the average detection accuracy of the proposed algorithm in the test set is 97.8%, the recognition speed is 37 frame/s and the computational quantity and parameter quantity of the model are 4.882 G and 8.851 M respectively. By testing the effects of different network models and changing the light intensity, the method embodies the advantages of high detection accuracy, fast speed, lightweight and the proposed algorithm has stronger adaptability and anti-interference ability.

Key words: longitudinal tear    multi-dimensional detection    MobileNetv3    Mixed domain attention mechanism    YOLOv4    lightweight
收稿日期: 2021-11-29 出版日期: 2022-12-02
CLC:  TP 391  
基金资助: 国家自然科学基金资助项目(51874004);国家重点研发计划资助项目(2020YFB1314203);安徽省重点研发计划资助项目(202004a07020043);芜湖市研究院研发专项基金资助项目(ALW2021YF10)
通讯作者: 胡坤     E-mail: 2698341084@qq.com;hk924@126.com
作者简介: 李飞(1995—),女,硕士生,从事带式输送机视觉检测研究. orcid.org/0000-0002-5432-8867. E-mail: 2698341084@qq.com
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章  
李飞
胡坤
张勇
王文善
蒋浩

引用本文:

李飞,胡坤,张勇,王文善,蒋浩. 基于混合域注意力YOLOv4的输送带纵向撕裂多维度检测[J]. 浙江大学学报(工学版), 2022, 56(11): 2156-2167.

Fei LI,Kun HU,Yong ZHANG,Wen-shan WANG,Hao JIANG. Multi-dimensional detection of longitudinal tearing of conveyor belt based on YOLOv4 of hybrid domain attention. Journal of ZheJiang University (Engineering Science), 2022, 56(11): 2156-2167.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2022.11.006        https://www.zjujournals.com/eng/CN/Y2022/V56/I11/2156

图 1  空间域注意力机制STNet
图 2  输送带纵向撕裂图像的仿射变换
图 3  高效通道注意力ECA模型
图 4  混合域注意力机制结构ECSNet
图 5  改进的Mobilenetv3网络模型(ECSMv3)
图 6  标准卷积和深度可分离卷积
层级名称 Input Out Numbers Activation
Function
Attention
Conv2D_BN_
hard-swish
4162×3 2082×16 1 hard-swish
Bneck_block 2082×16 2082×16 1 relu
Bneck_block 2082×16 1042×24 2 relu
Bneck_block 1042×24 522×40 3 relu
Bneck_block 522×40 262×112 6 hard-swish
Bneck_block 262×112 132×160 3 hard-swish
表 1  融入混合域注意力机制的MobileNetv3网络结构(ECSMv3)
图 7  ECSMv3-YOLOv4网络模型结构
图 8  相机与光源安装位置图
图 9  输送带撕裂检测流程图
图 10  输送带纵向撕裂识别检测实验装置
图 11  输送带纵向撕裂图像增强的样本数据
图 12  ECSMv3-YOLOv4训练模型损失曲线
网络模型 主干网络 FLOPs/G Params/M mAP FPS/(frame·s?1
YOLOv4 CSPDarknet-53 59.765 63.943 0.939 31
YOLOv4 DenseNet-169 48.654 50.171 0.925 22
YOLOv4 ResNet-50 53.948 61.522 0.912 33
YOLOv4 VGG16 130.936 51.778 0.897 32
YOLOv4 MobileNetv3 26.115 39.570 0.953 33
表 2  YOLOv4不同的主干网络之间的测试结果
网络模型 主干网络 FLOPs/G Params/M mAP FPS/(frame·s?1
YOLOv4 MobileNetv3+SENet 7.030 11.309 0.952 33
YOLOv4 MobileNetv3+ECA 7.027 9.797 0.958 35
YOLOv4 MobileNetv3+CBAM 7.032 10.550 0.963 33
YOLOv4 MobileNetv3+ECSNet 7.029 9.798 0.976 34
表 3  改变注意力机制的网络性能测试结果
网络模型 主干网络 FLOPs/G Params/M mAP FPS/(frame·s?1
ECSMv3_YOLOv4+3Heads+9Anchors ECSMv3 7.029 9.798 0.976 34
ECSMv3_YOLOv4+3Heads+6Anchors ECSMv3 7.012 9.785 0.968 35
ECSMv3_YOLOv4+2Heads+6Anchors ECSMv3 4.882 8.851 0.978 37
ECSMv3_YOLOv4+2Heads+4Anchors ECSMv3 4.875 8.841 0.951 38
表 4  不同数量的Prediction Heads和Anchors对模型精度的测试结果
网络模型 主干网络 FLOPs/G Params/M mAP FPS/(frame·s?1
YOLOv3 Darknet-53 65.527 61.529 0.901 32
YOLOv4-Tiny CSPdarknet53-Tiny 6.823 5.876 0.875 45
SSD MobileNetv2 2.493 3.675 0.884 33
YOLOX Focus+CSPDarknet 11.254 8.938 0.952 35
ECSMv3_YOLOv4 MobileNetv3+ECSNet 4.882 8.851 0.978 37
表 5  不同网络模型之间的测试效果对比
图 13  不同网络模型对输送带纵向撕裂多维度检测效果图
图 14  不同光照强度的ECSMv3_YOLOv4模型检测结果
1 杨小林, 葛世荣, 祖洪斌, 等 带式输送机永磁智能驱动系统及其控制策略[J]. 煤炭学报, 2020, 45 (6): 2116- 2126
YANG Xiao-lin, GE Shi-rong, ZU Hong-bin, et al The permanent magnet intelligent drive system of belt conveyor and its control strategy[J]. Journal of China Coal Society, 2020, 45 (6): 2116- 2126
doi: 10.13225/j.cnki.jccs.zn20.0345
2 PETRIKOVA I, MARVALOVA B, SAMAL S, et al Digital image correlation as a measurement tool for large deformations of a conveyor belt[J]. Applied Mechanics and Materials, 2015, 732: 77- 80
doi: 10.4028/www.scientific.net/AMM.732.77
3 曹虎奇 煤矿带式输送机撕带断带研究分析[J]. 煤炭科学技术, 2015, 43 (Suppl.2): 130- 134
CAO Hu-qi Research and analysis on tearing and breaking belt of coal mine belt conveyor[J]. Coal Science and Technology, 2015, 43 (Suppl.2): 130- 134
4 刘伟力, 乔铁柱 矿用输送带纵向撕裂检测系统研究[J]. 工矿自动化, 2017, 43 (2): 78- 81
LIU Wei-li, QIAO Tie-zhu Research on longitudinal tear detection system of mine conveyor belt[J]. Industrial and Mining Automation, 2017, 43 (2): 78- 81
doi: 10.13272/j.issn.1671-251x.2017.02.017
5 PANG Y S, LODEWIJKS G. A novel embedded conductive detection system for intelligent conveyor belt monitoring [C]// IEEE International Conference on Service Operations and Logistics and Informatics. Shanghai: IEEE, 2006: 803-808.
6 LI X G, SHEN L F, MING Z X, et al Laser-based online machine vision detection for longitudinal rip of conveyor belt[J]. Optik, 2018, 168: 360- 369
doi: 10.1016/j.ijleo.2018.04.053
7 BLAZEJ R, JURDZIAK L, KOZLOWSKI T, et al The use of magnetic sensors in monitoring the condition of the core in steel cord conveyor belts-Tests of the measuring probe and the design of the diag belt system.[J]. Measurement, 2018, 123: 48- 53
doi: 10.1016/j.measurement.2018.03.051
8 YANG R Y, QIAO T Z, PANG Y S, et al Infrared spectrum analysis method for detection and early warning of longitudinal tear of mine conveyor belt[J]. Measurement, 2020, 165: 107856
doi: 10.1016/j.measurement.2020.107856
9 HOU C C, QIAO T Z, ZHANG H T, et al Multispectral visual detection method for conveyor belt longitudinal tear[J]. Measurement, 2019, 143: 246- 257
doi: 10.1016/j.measurement.2019.05.010
10 王志星. 输送带纵向撕裂双目视觉在线检测系统研究与设计[D]. 太原: 太原理工大学, 2018: 33-40.
WANG Zhi-xing. Research and design of binocular vision online detection system for longitudinal tearing of conveyor belt [D]. Taiyuan: Taiyuan University of Technology, 2018: 33-40.
11 刘伟力. 输送带纵向撕裂机器视觉在线监控系统研究[D]. 太原: 太原理工大学, 2017: 39-46.
LIU Wei-li. Research on online monitoring system of conveyor belt longitudinal tearing based on machine vision [D]. Taiyuan: Taiyuan University of Technology, 2017: 39-46.
12 LI W W, LI C Q, YAN F L Research on belt tear detection algorithm based on multiple sets of laser line assistance[J]. Measurement, 2021, 174 (2): 109047
13 GIRSHICK R. Fast R-CNN [EB/OL]. [2021-09-15]. https://arxiv.org/abs/1504.08083.
14 HE K M, GKIOXARI G, DOLLAR P, et al. Mask R-CNN [EB/OL]. [2021-09-15]. https://arxiv.org/abs/1703.06870.
15 LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot multibox detector [C]// European Conference on Computer Vision. Berlin: Springer, 2016: 21-37.
16 REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection [C]// IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 779-788.
17 YANG J, LI S B, WANG Z, et al Real-time tiny part defect detection system in manufacturing using deep learning[J]. IEEE Access, 2019, 7 (1): 89278- 89291
18 LI Z Y, ZHU X N, ZHOU J. Intelligent monitoring system of coal conveyor belt based on computer vision technology [C]// International Conference on Dependable Systems and Their Applications. Harbin: IEEE, 2020: 359-364.
19 蒋镕圻, 彭月平, 谢文宣, 等 嵌入scSE模块的改进YOLOv4小目标检测算法[J]. 图学学报, 2021, 42 (4): 546- 555
JIANG Rong-qi, PENG Yue-ping, XIE Wen-xuan, et al Improved YOLOv4 small target detection algorithm embedded with scSE module[J]. Journal of Graphics, 2021, 42 (4): 546- 555
20 WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module [C]// European Conference on Computer Vision. Berlin: Springer, 2018: 3-19.
21 HOWARD A G, ZHU M, CHEN B, et al. MobileNets: efficient convolutional neural networks for mobile vision applications [EB/OL]. [2021-09-18]. https://arxiv.org/abs /1704.04861.
22 ZHANG X, ZHOU X, LIN M, et al. ShuffleNet: an extremely efficient convolutional neural network for mobile devices [EB/OL]. [2021-09-18]. https://arxiv.org/abs/1707. 01083v2.
23 薄景文, 张春堂, 樊春玲, 等 改进YOLOv3的矿石输送带杂物检测方法[J]. 计算机工程与应用, 2021, 57 (21): 248- 255
BO Jing-wen, ZHANG Chun-tang, FAN Chun-ling, et al Improved YOLOv3 method for detecting trash on ore conveyor belts[J]. Computer Engineering and Applications, 2021, 57 (21): 248- 255
doi: 10.3778/j.issn.1002-8331.2105-0025
24 周宇杰, 徐善永, 黄友锐, 等 基于改进YOLOv4的输送带损伤检测方法[J]. 工矿自动化, 2021, 47 (11): 61- 65
ZHOU Yu-jie, XU Shan-yong, HUANG You-rui, et al Conveyor belt damage detection method based on improved YOLOv4[J]. Industry and Mine Automation, 2021, 47 (11): 61- 65
doi: 10.13272/j.issn.1671-251x.17843
25 JADERBERG M, KAREN S, ANDREW Z. Spatial transformer networks [J]. Advances in Neural Information Processing Systems, 2015 (28): 2017-2025.
26 WANG Q L, WU B G, ZHU P F, et al. ECA-Net: efficient channel attention for deep convolutional neural networks [C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 11531-11539.
27 WANG X L, GIRSHICK R, GUPTA A, et al. Non-local neural networks [C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 7794-7803.
[1] 郝琨,王阔,王贝贝. 基于改进Mobilenet-YOLOv3的轻量级水下生物检测算法[J]. 浙江大学学报(工学版), 2022, 56(8): 1622-1632.
[2] 褚晶辉,史李栋,井佩光,吕卫. 适用于目标检测的上下文感知知识蒸馏网络[J]. 浙江大学学报(工学版), 2022, 56(3): 503-509.
[3] 辛文斌,郝惠敏,卜明龙,兰媛,黄家海,熊晓燕. 基于ShuffleNetv2-YOLOv3模型的静态手势实时识别方法[J]. 浙江大学学报(工学版), 2021, 55(10): 1815-1824.
[4] 童水光,何顺,童哲铭,李元松,徐征宇,方晓晖,谭达辉,钟玉伟. 基于导重法的叉车门架轻量化设计[J]. 浙江大学学报(工学版), 2020, 54(1): 33-39.
[5] 柯俊,史文库,钱琛,李国民,袁可. 复合材料板簧刚度的预测及匹配设计方法[J]. 浙江大学学报(工学版), 2015, 49(11): 2103-2110.
[6] 杨陈, 郝志勇, 刘保林, 等. 变密度法在气缸体轻量化设计中的应用[J]. J4, 2009, 43(5): 916-919.
[7] 彭禹 郝志勇. 基于虚拟样机的动态轻量化设计方法[J]. J4, 2008, 42(6): 984-988.
[8] 贾维新 郝志勇 徐红梅. 基于结构优化方法的单缸机机体轻量化设计[J]. J4, 2008, 42(2): 224-228.