Please wait a minute...
Journal of ZheJiang University (Engineering Science)  2022, Vol. 56 Issue (11): 2156-2167    DOI: 10.3785/j.issn.1008-973X.2022.11.006
    
Multi-dimensional detection of longitudinal tearing of conveyor belt based on YOLOv4 of hybrid domain attention
Fei LI1,2(),Kun HU1,2,3,*(),Yong ZHANG2,Wen-shan WANG1,Hao JIANG1
1. School of Mechanical Engineering, Anhui University of Science and Technology, Huainan 232001, China
2. State Key Laboratory of Mining Response and Disaster Prevention and Control in Deep Coal Mines, Anhui University of Science and Technology, Huainan 232001, China
3. Institute of Environment-friendly Materials and Occupational Health, Anhui University of Science and Technology, Wuhu, 241003, China
Download: HTML     PDF(8267KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

An efficient MobileNetv3 and YOLOv4 integrated network multi-dimensional real-time detection method for longitudinal tearing of conveyor belt was proposed to aim at the problem of single dimension and high complexity of model in the detection of the longitudinal tearing target of the conveyor belt. The lightweight network MobileNetv3 based on the object detection algorithm of YOLOv4 was used to replace CSPDarknet53 as the backbone network of YOLOv4. The ECSNet was constructed by combining efficient channel domain ECA model and spatial transformer network (STNet). The ECSNet was embedded in MobileNetv3 to improve the attention of model to space and channels. The deep separable convolution block was introduced to replace the 3*3 convolution in the network and the Prediction Heads of YOLOv4 were reduced to two scales. The network model was lightened, the complexity and training difficulty were reduced and ECSMv3_YOLOv4 model was built. The K-means was used to cluster six Anchors to predict the height and width of the bounding box, which improved the detection performance of the network for surface tearing. The multi-dimensional intelligent inspection prototype of belt conveyor was developed, the longitudinal tear data set of multi-dimensional surface of conveyor belt was collected and made. The training, testing, identification and positioning experiments of network model were carried out. The results show that the average detection accuracy of the proposed algorithm in the test set is 97.8%, the recognition speed is 37 frame/s and the computational quantity and parameter quantity of the model are 4.882 G and 8.851 M respectively. By testing the effects of different network models and changing the light intensity, the method embodies the advantages of high detection accuracy, fast speed, lightweight and the proposed algorithm has stronger adaptability and anti-interference ability.



Key wordslongitudinal tear      multi-dimensional detection      MobileNetv3      Mixed domain attention mechanism      YOLOv4      lightweight     
Received: 29 November 2021      Published: 02 December 2022
CLC:  TP 391  
Fund:  国家自然科学基金资助项目(51874004);国家重点研发计划资助项目(2020YFB1314203);安徽省重点研发计划资助项目(202004a07020043);芜湖市研究院研发专项基金资助项目(ALW2021YF10)
Corresponding Authors: Kun HU     E-mail: 2698341084@qq.com;hk924@126.com
Cite this article:

Fei LI,Kun HU,Yong ZHANG,Wen-shan WANG,Hao JIANG. Multi-dimensional detection of longitudinal tearing of conveyor belt based on YOLOv4 of hybrid domain attention. Journal of ZheJiang University (Engineering Science), 2022, 56(11): 2156-2167.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2022.11.006     OR     https://www.zjujournals.com/eng/Y2022/V56/I11/2156


基于混合域注意力YOLOv4的输送带纵向撕裂多维度检测

针对输送带纵向撕裂目标检测维度单一、模型复杂度高等问题,提出一种高效的MobileNetv3及YOLOv4集成网络输送带纵向撕裂多维度实时检测方法. 基于YOLOv4目标识别算法,通过将轻量化网络MobileNetv3代替CSPDarknet53作为骨干网络,结合高效通道域ECA模块和空间域注意力机制(STNet)构建混合域注意力网络(ECSNet),改进了MobileNetv3嵌入ECSNet,并且提升了模型对空间和通道的关注度. 引入深度可分离卷积块代替网络中3*3卷积,并将YOLOv4的检测头(Prediction Heads)缩减为2种尺度,轻量化模型降低网络复杂度和训练难度,完成ECSMv3_YOLOv4模型的搭建,使用K-means聚类6个Anchors预测目标框高宽,提高网络对表面撕裂的检测性能. 研制带式输送机多维度智能巡检样机,采集制作输送带多维度面的纵向撕裂数据集,开展网络模型的训练、测试、识别和定位实验. 结果表明,提出算法在测试集中的平均识别准确率为97.8%,识别速度为37 帧/s,模型的计算量和参数量为4.882 G和8.851 M,通过试验不同的网络模型效果和改变光照强度,该方法体现出检测精度高、速度快和轻量化等优点,具备更强的适应性和抗干扰能力.


关键词: 纵向撕裂,  多维度检测,  MobileNetv3,  混合域注意力机制,  YOLOv4,  轻量化 
Fig.1 Spatial attention mechanisms
Fig.2 Affine transformation of longitudinal tearing image of conveyor belt
Fig.3 Efficient channel attention ECA model
Fig.4 Structure of hybrid domain attention mechanism
Fig.5 Improved Mobilenetv3 network model (ECSMv3)
Fig.6 Standard convolution and depth wise separable convolution
层级名称 Input Out Numbers Activation
Function
Attention
Conv2D_BN_
hard-swish
4162×3 2082×16 1 hard-swish
Bneck_block 2082×16 2082×16 1 relu
Bneck_block 2082×16 1042×24 2 relu
Bneck_block 1042×24 522×40 3 relu
Bneck_block 522×40 262×112 6 hard-swish
Bneck_block 262×112 132×160 3 hard-swish
Tab.1 MobileNetv3 network architecture with mixed domain attention mechanism
Fig.7 ECSMv3-YOLOv4 network model structure
Fig.8 Installation position diagram of cameras and light source
Fig.9 Flow chart of conveyor belt tear detection
Fig.10 Experimental device of conveyor belt longitudinal tear identification and detection
Fig.11 Sample data of longitudinal tearing image enhancement of conveyor belt
Fig.12 Ecsmv3-yolov4 training model loss function curve
网络模型 主干网络 FLOPs/G Params/M mAP FPS/(frame·s?1
YOLOv4 CSPDarknet-53 59.765 63.943 0.939 31
YOLOv4 DenseNet-169 48.654 50.171 0.925 22
YOLOv4 ResNet-50 53.948 61.522 0.912 33
YOLOv4 VGG16 130.936 51.778 0.897 32
YOLOv4 MobileNetv3 26.115 39.570 0.953 33
Tab.2 Results of YOLOv4 different backbone networks
网络模型 主干网络 FLOPs/G Params/M mAP FPS/(frame·s?1
YOLOv4 MobileNetv3+SENet 7.030 11.309 0.952 33
YOLOv4 MobileNetv3+ECA 7.027 9.797 0.958 35
YOLOv4 MobileNetv3+CBAM 7.032 10.550 0.963 33
YOLOv4 MobileNetv3+ECSNet 7.029 9.798 0.976 34
Tab.3 Network performance test results for changing attention mechanism
网络模型 主干网络 FLOPs/G Params/M mAP FPS/(frame·s?1
ECSMv3_YOLOv4+3Heads+9Anchors ECSMv3 7.029 9.798 0.976 34
ECSMv3_YOLOv4+3Heads+6Anchors ECSMv3 7.012 9.785 0.968 35
ECSMv3_YOLOv4+2Heads+6Anchors ECSMv3 4.882 8.851 0.978 37
ECSMv3_YOLOv4+2Heads+4Anchors ECSMv3 4.875 8.841 0.951 38
Tab.4 Different numbers of Prediction Heads and Anchors test results for model accuracy
网络模型 主干网络 FLOPs/G Params/M mAP FPS/(frame·s?1
YOLOv3 Darknet-53 65.527 61.529 0.901 32
YOLOv4-Tiny CSPdarknet53-Tiny 6.823 5.876 0.875 45
SSD MobileNetv2 2.493 3.675 0.884 33
YOLOX Focus+CSPDarknet 11.254 8.938 0.952 35
ECSMv3_YOLOv4 MobileNetv3+ECSNet 4.882 8.851 0.978 37
Tab.5 Effect comparison of different network models
Fig.13 Detection effect diagram of longitudinal tearing of multi-dimensional conveyor belt by different network models
Fig.14 Detection results of ECSMv3_YOLOv4 model with different light intensities
[1]   杨小林, 葛世荣, 祖洪斌, 等 带式输送机永磁智能驱动系统及其控制策略[J]. 煤炭学报, 2020, 45 (6): 2116- 2126
YANG Xiao-lin, GE Shi-rong, ZU Hong-bin, et al The permanent magnet intelligent drive system of belt conveyor and its control strategy[J]. Journal of China Coal Society, 2020, 45 (6): 2116- 2126
doi: 10.13225/j.cnki.jccs.zn20.0345
[2]   PETRIKOVA I, MARVALOVA B, SAMAL S, et al Digital image correlation as a measurement tool for large deformations of a conveyor belt[J]. Applied Mechanics and Materials, 2015, 732: 77- 80
doi: 10.4028/www.scientific.net/AMM.732.77
[3]   曹虎奇 煤矿带式输送机撕带断带研究分析[J]. 煤炭科学技术, 2015, 43 (Suppl.2): 130- 134
CAO Hu-qi Research and analysis on tearing and breaking belt of coal mine belt conveyor[J]. Coal Science and Technology, 2015, 43 (Suppl.2): 130- 134
[4]   刘伟力, 乔铁柱 矿用输送带纵向撕裂检测系统研究[J]. 工矿自动化, 2017, 43 (2): 78- 81
LIU Wei-li, QIAO Tie-zhu Research on longitudinal tear detection system of mine conveyor belt[J]. Industrial and Mining Automation, 2017, 43 (2): 78- 81
doi: 10.13272/j.issn.1671-251x.2017.02.017
[5]   PANG Y S, LODEWIJKS G. A novel embedded conductive detection system for intelligent conveyor belt monitoring [C]// IEEE International Conference on Service Operations and Logistics and Informatics. Shanghai: IEEE, 2006: 803-808.
[6]   LI X G, SHEN L F, MING Z X, et al Laser-based online machine vision detection for longitudinal rip of conveyor belt[J]. Optik, 2018, 168: 360- 369
doi: 10.1016/j.ijleo.2018.04.053
[7]   BLAZEJ R, JURDZIAK L, KOZLOWSKI T, et al The use of magnetic sensors in monitoring the condition of the core in steel cord conveyor belts-Tests of the measuring probe and the design of the diag belt system.[J]. Measurement, 2018, 123: 48- 53
doi: 10.1016/j.measurement.2018.03.051
[8]   YANG R Y, QIAO T Z, PANG Y S, et al Infrared spectrum analysis method for detection and early warning of longitudinal tear of mine conveyor belt[J]. Measurement, 2020, 165: 107856
doi: 10.1016/j.measurement.2020.107856
[9]   HOU C C, QIAO T Z, ZHANG H T, et al Multispectral visual detection method for conveyor belt longitudinal tear[J]. Measurement, 2019, 143: 246- 257
doi: 10.1016/j.measurement.2019.05.010
[10]   王志星. 输送带纵向撕裂双目视觉在线检测系统研究与设计[D]. 太原: 太原理工大学, 2018: 33-40.
WANG Zhi-xing. Research and design of binocular vision online detection system for longitudinal tearing of conveyor belt [D]. Taiyuan: Taiyuan University of Technology, 2018: 33-40.
[11]   刘伟力. 输送带纵向撕裂机器视觉在线监控系统研究[D]. 太原: 太原理工大学, 2017: 39-46.
LIU Wei-li. Research on online monitoring system of conveyor belt longitudinal tearing based on machine vision [D]. Taiyuan: Taiyuan University of Technology, 2017: 39-46.
[12]   LI W W, LI C Q, YAN F L Research on belt tear detection algorithm based on multiple sets of laser line assistance[J]. Measurement, 2021, 174 (2): 109047
[13]   GIRSHICK R. Fast R-CNN [EB/OL]. [2021-09-15]. https://arxiv.org/abs/1504.08083.
[14]   HE K M, GKIOXARI G, DOLLAR P, et al. Mask R-CNN [EB/OL]. [2021-09-15]. https://arxiv.org/abs/1703.06870.
[15]   LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot multibox detector [C]// European Conference on Computer Vision. Berlin: Springer, 2016: 21-37.
[16]   REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection [C]// IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 779-788.
[17]   YANG J, LI S B, WANG Z, et al Real-time tiny part defect detection system in manufacturing using deep learning[J]. IEEE Access, 2019, 7 (1): 89278- 89291
[18]   LI Z Y, ZHU X N, ZHOU J. Intelligent monitoring system of coal conveyor belt based on computer vision technology [C]// International Conference on Dependable Systems and Their Applications. Harbin: IEEE, 2020: 359-364.
[19]   蒋镕圻, 彭月平, 谢文宣, 等 嵌入scSE模块的改进YOLOv4小目标检测算法[J]. 图学学报, 2021, 42 (4): 546- 555
JIANG Rong-qi, PENG Yue-ping, XIE Wen-xuan, et al Improved YOLOv4 small target detection algorithm embedded with scSE module[J]. Journal of Graphics, 2021, 42 (4): 546- 555
[20]   WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module [C]// European Conference on Computer Vision. Berlin: Springer, 2018: 3-19.
[21]   HOWARD A G, ZHU M, CHEN B, et al. MobileNets: efficient convolutional neural networks for mobile vision applications [EB/OL]. [2021-09-18]. https://arxiv.org/abs /1704.04861.
[22]   ZHANG X, ZHOU X, LIN M, et al. ShuffleNet: an extremely efficient convolutional neural network for mobile devices [EB/OL]. [2021-09-18]. https://arxiv.org/abs/1707. 01083v2.
[23]   薄景文, 张春堂, 樊春玲, 等 改进YOLOv3的矿石输送带杂物检测方法[J]. 计算机工程与应用, 2021, 57 (21): 248- 255
BO Jing-wen, ZHANG Chun-tang, FAN Chun-ling, et al Improved YOLOv3 method for detecting trash on ore conveyor belts[J]. Computer Engineering and Applications, 2021, 57 (21): 248- 255
doi: 10.3778/j.issn.1002-8331.2105-0025
[24]   周宇杰, 徐善永, 黄友锐, 等 基于改进YOLOv4的输送带损伤检测方法[J]. 工矿自动化, 2021, 47 (11): 61- 65
ZHOU Yu-jie, XU Shan-yong, HUANG You-rui, et al Conveyor belt damage detection method based on improved YOLOv4[J]. Industry and Mine Automation, 2021, 47 (11): 61- 65
doi: 10.13272/j.issn.1671-251x.17843
[25]   JADERBERG M, KAREN S, ANDREW Z. Spatial transformer networks [J]. Advances in Neural Information Processing Systems, 2015 (28): 2017-2025.
[26]   WANG Q L, WU B G, ZHU P F, et al. ECA-Net: efficient channel attention for deep convolutional neural networks [C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 11531-11539.
[27]   WANG X L, GIRSHICK R, GUPTA A, et al. Non-local neural networks [C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 7794-7803.
[1] Kun HAO,Kuo WANG,Bei-bei WANG. Lightweight underwater biological detection algorithm based on improved Mobilenet-YOLOv3[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(8): 1622-1632.
[2] Jiang-bo XU,Yuan-zhi WANG,Yu QI,Bao-hua CAO,Yong-zhen LUO,Chang-gen YAN,Xiao-hua YANG,Han BAO,Yu-zhou XIANG. Deformation characteristics of fiber-reinforced foam lightweight soil under cyclic loading and unloading[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(1): 111-117.
[3] Wen-bin XIN,Hui-min HAO,Ming-long BU,Yuan LAN,Jia-hai HUANG,Xiao-yan XIONG. Static gesture real-time recognition method based on ShuffleNetv2-YOLOv3 model[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(10): 1815-1824.
[4] Jun LUO,Xu-dong SHAO,Jun-hui CAO,Wei FAN,Bi-da PEI. Orthogonal test and calculation method of cracking load of steel-ultra-high performance concrete composite specimen[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(5): 909-920.
[5] Shui-guang TONG,Shun HE,Zhe-ming TONG,Yuan-song LI,Zheng-yu XU,Xiao-hui FANG,Da-hui TAN,Yu-wei ZHONG. Lightweight design of forklift frame based on guide weight method[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(1): 33-39.
[6] MA Yun-fei, WANG Tao, CHEN Hao, ZHANG Fan, LOU Xiao-xuan, XU Lu-min, YANG Wen-bing. Fault-cube attack on SIMON family of lightweight block ciphers[J]. Journal of ZheJiang University (Engineering Science), 2017, 51(9): 1770-1779.
[7] HAO Zi-yi, XIANG Xiao-yan, CHEN Chen, MENG Jian-yi. Error cancellation flip-flop design with lightweight in-situ error correction[J]. Journal of ZheJiang University (Engineering Science), 2017, 51(3): 605-611.