Please wait a minute...
浙江大学学报(工学版)  2024, Vol. 58 Issue (2): 370-380    DOI: 10.3785/j.issn.1008-973X.2024.02.015
机械工程     
基于改进YOLOv8s的鼓形滚子表面缺陷检测算法
王安静1(),袁巨龙1,*(),朱勇建2,陈聪1,吴金津1
1. 浙江工业大学 机械工程学院,浙江 杭州 310023
2. 宁波敏捷信息科技有限公司,浙江 慈溪 315300
Drum roller surface defect detection algorithm based on improved YOLOv8s
Anjing WANG1(),Julong YUAN1,*(),Yongjian ZHU2,Cong CHEN1,Jinjin WU1
1. College of Mechanical Engineering, Zhejiang University of Technology, Hangzhou 310023, China
2. Ningbo Agile Information Technology Limited Company, Cixi 315300, China
 全文: PDF(3310 KB)   HTML
摘要:

为了提高鼓形滚子表面微小瑕疵缺陷检测的精确率和召回率,增强模型对小目标缺陷的检测能力,针对YOLOv8s网络,提出细粒化卷积模块SPD-Conv来代替卷积下采样,细粒化地提取小缺陷的特征. 在特征融合模块,引入GFPN特征融合模块,增强相邻层级间的跨尺度连接和同尺度下的跨层连接,有助于小目标特征信息在卷积网络的传递. 在头部增加小目标检测层,提高模型对小缺陷的检测能力. 在损失函数方面,利用动态非单调聚焦的Wise-IOU的边界框损失函数替换CIOU,在加快网络收敛的同时,提高网络检测的精度. 在自制的鼓形滚子缺陷数据集上进行测试,结果表明,改进的YOLOv8s在倒角数据集、侧面数据集、端面数据集的mAP@0.5分别达到0.911、0.983、0.935,相比于YOLOv8s,mAP@0.5分别提高了6.4%、3.3%、4%,精确度和召回率也有一定的提升,平均每张图片的检测时间为23 ms. 与原模型相比,改进的YOLOv8s对小目标缺陷有更好的定位能力和检测精度,检测速度能够满足工业大批量检测的要求.

关键词: 鼓形滚子缺陷检测YOLOv8s细粒化卷积广义的特征金字塔网络(GFPN)Wise-IOU    
Abstract:

A fine-grained convolution module SPD-Conv was proposed to replace the convolution subsampling for YOLOv8s network and extract the features of small defects in a fine-grained way in order to improve the accuracy and recall rate of the detection of small defects on the surface of drum rollers and enhance the detection ability of the model for small target defects. GFPN feature fusion module was introduced to enhance the cross-scale connection between adjacent layers and cross-layer connection under the same scale in the feature fusion module, which is conducive to the transmission of small target feature information in the convolutional network. The small target detection layer was added to the head in order to improve the detection ability of the model. The boundary frame loss function of Wise-IOU was used to replace CIOU in terms of loss function, which could accelerate network convergence and improve the accuracy of network detection. The test was conducted on the self-made drum roller defect dataset. Results showed that the improved YOLOv8s achieved 0.911, 0.983 and 0.935 in the chamfer dataset, side dataset and end dataset, respectively. mAP@0.5 increased by 6.4%, 3.3% and 4% respectively compared with YOLOv8s. Accuracy and recall rates have improved with an average detection time of 23 ms per image. The improved YOLOv8s has better localization ability and detection accuracy for small target defects compared with the original model, and the detection speed can meet the requirements of industrial mass detection.

Key words: drum roller    defect detection    YOLOv8s    fine-grained convolution    general feature pyramid network (GFPN)    Wise-IOU
收稿日期: 2023-07-18 出版日期: 2024-01-23
CLC:  TP 391  
基金资助: NSFC-浙江两化融合联合基金资助项目(U1809221);宁波泛3315人才项目资助项目
通讯作者: 袁巨龙     E-mail: 2082673638@qq.com;jlyuan@zjut.edu.cn
作者简介: 王安静(1999—),男,硕士生,从事机器视觉的研究. orcid.org/0009-0008-6252-3096. E-mail:2082673638@qq.com
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章  
王安静
袁巨龙
朱勇建
陈聪
吴金津

引用本文:

王安静,袁巨龙,朱勇建,陈聪,吴金津. 基于改进YOLOv8s的鼓形滚子表面缺陷检测算法[J]. 浙江大学学报(工学版), 2024, 58(2): 370-380.

Anjing WANG,Julong YUAN,Yongjian ZHU,Cong CHEN,Jinjin WU. Drum roller surface defect detection algorithm based on improved YOLOv8s. Journal of ZheJiang University (Engineering Science), 2024, 58(2): 370-380.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2024.02.015        https://www.zjujournals.com/eng/CN/Y2024/V58/I2/370

图 1  YOLOv8s的网络结构
图 2  卷积下采样模块
图 3  SPD-Conv(scale = 2)模块的结构图
图 4  PANet特征融合
图 5  GFPN特征融合
图 6  自适应特征融合的过程
图 7  四检测头模型的示意图
图 8  CIOU参数的示意图
图 9  Wise-IOU参数的示意图
图 10  改进的YOLOv8s模型
图 11  鼓形滚子图片
图 12  鼓形滚子缺陷检测平台
图 13  倒角缺陷的细节图
图 14  端面缺陷的细节图
图 15  侧面缺陷的细节图
配置环境配置名称(版本)
操作系统Windows10
CPUInter(R) Xeon(R) Gold 5218 (2.3 GHz)
GPUNVIDIA Geforce RTX 2080Ti(11 GB)×2
编译器Python-3.9
深度学习框架Pytorch-1.10.0
加速模块CUDA Toolkit-11.3.1
表 1  深度学习的实验环境配置
参数含义数值
images size图像尺度640
batch size批数量16
E迭代次数300
lr学习率0.01
Momentum动量0.937
Weight_decay权重衰减参数0.000 5
表 2  深度学习的实验超参数
真实情况预测结果
正例反例
正例TPFN
反例FPTN
表 3  分类结果的混淆矩阵
算法APPRmAP@0.5v/(帧·s?1)Np/107
缺失剥落磕碰
YOLOv8s0.9160.9200.7050.8500.8220.84780.61.112
YOLOv8s+小目标检测层0.9590.9370.7980.9000.8470.89867.91.263
YOLOv8s+GFPN0.9360.9130.7290.8640.8300.85976.91.347
YOLOv8s+SPD-Conv0.9420.9420.7410.8600.8540.87577.21.266
YOLOv8s+3个改进点0.9610.9410.8100.9110.8600.90157.71.410
表 4  倒角检测结果
算法APPRmAP@0.5v/(帧·s?1)Np/107
划痕环伤锈蚀点蚀
YOLOv8s0.9760.9940.9560.9000.9500.9200.95063.41.112
YOLOv8s+小目标检测层0.9870.9930.9710.9510.9610.9480.97154.11.263
YOLOv8s+GFPN0.9830.9940.9640.9160.9590.9330.96459.21.347
YOLOv8s+SPD-Conv0.9740.9950.9650.9100.9410.9340.96251.61.266
YOLOv8s+3个改进点0.9890.9940.9710.9600.9680.9420.97543.41.410
表 5  侧面检测结果
算法APPRmAP@0.5v/(帧·s?1)Np/107
环伤磕碰锈蚀点蚀崩边
YOLOv8s0.9760.9100.9150.7010.9850.9010.8500.89560.51.112
YOLOv8s+小目标检测层0.9820.8950.9280.7740.9920.9100.8730.91451.61.263
YOLOv8+GFPN0.9730.9240.9350.7240.9940.9120.8670.90857.11.347
YOLOv8+SPD-Conv0.9670.9430.9290.7160.9950.9080.8630.91053.41.266
YOLOv8s+3个改进点0.9810.9510.9450.8010.9950.9230.8810.93040.21.410
表 6  端面检测结果
图 16  侧面损失函数变化图
图 17  倒角损失函数变化图
图 18  端面损失函数变化图
检测模型SPD-ConvGFPN小目标
检测层
LWIOUV3mAP@0.5v/
(帧·s?1)
Model1××××0.84780.6
Model2×××0.87577.2
Model3××0.88569.3
Model4×0.90157.7
Model50.91147.7
表 7  倒角消融的实验结果
检测模型SPD-ConvGFPN小目标
检测层
LWIOUV3mAP@0.5v/
(帧·s?1)
Model1××××0.95061.4
Model2×××0.96251.6
Model3××0.96947.8
Model4×0.97543.4
Model50.98340.7
表 8  侧面消融的实验结果
检测模型SPD-ConvGFPN小目标
检测层
LWIOUV3mAP@0.5v/
(帧·s?1)
Model1××××0.89558.5
Model2×××0.91053.4
Model3××0.92447.8
Model4×0.93040.2
Model50.93538.0
表 9  端面消融的实验结果
算法模型Np/106mAP@0.5v/(帧·s?1)
Faster R-CNN136.100.85115.4
SSD24.010.54050.1
YOLOv361.530.74051.8
YOLOv463.950.79054.7
YOLOv5s7.030.82981.9
YOLOv737.210.81065.5
YOLOv8s11.200.84780.6
本文方法14.100.91147.7
表 10  倒角模型的对比结果
算法模型Np/106mAP@0.5v/(帧·s?1)
Faster R-CNN136.100.948.01
SSD24.010.6738.7
YOLOv361.530.8428.3
YOLOv463.950.8745.6
YOLOv5s7.030.9367.2
YOLOv737.210.9346.1
YOLOv8s11.200.9561.4
本文方法14.100.98340.7
表 11  侧面模型的对比结果
算法模型Np/106mAP@0.5v/(帧·s?1)
Faster R-CNN136.100.8529.4
SSD24.010.47245.6
YOLOv361.530.76547.1
YOLOv463.950.84749.8
YOLOv5s7.030.86350.9
YOLOv737.210.85636.7
YOLOv8s11.200.89558.5
本文方法14.100.93538.0
表 12  端面模型的对比结果
检测模型漏检率误检率
YOLOv8s8%11%
本文方法2%5%
表 13  在线检测结果的对比
图 19  YOLOv8s改进前、后的滚子缺陷表面检测结果对比图
算法mAP@0.5
点蚀划痕斑块轧入氧化皮
YOLOv8s0.8790.8860.8920.639
本文方法0.9090.9510.9180.651
表 14  东北大学热轧带钢表面缺陷数据集的对比结果
1 王姮, 卜燕, 张华, 等 基于分形维数的磁痕图像缺陷检测[J]. 计算机应用研究, 2015, 32 (2): 603- 605
WANG Heng, BU Yan, ZHANG Hua, et al Defect detection of magnetic trace image based on fractal dimension[J]. Application Research of Computers, 2015, 32 (2): 603- 605
doi: 10.3969/j.issn.1001-3695.2015.02.063
2 梁子千, 玄文博, 王婷, 等 基于二维阻抗特征的管道环焊缝缺陷涡流检测[J]. 仪器仪表学报, 2017, 38 (9): 2138- 2145
LIANG Ziqian, XUAN Wenbo, WANG Ting, et al Eddy current detection of pipe girth weld defects based on two-dimensional impedance characteristics[J]. Chinese Journal of Scientific Instrument, 2017, 38 (9): 2138- 2145
doi: 10.3969/j.issn.0254-3087.2017.09.006
3 陈振华, 郑志远, 卢超 不锈钢焊缝中超声传播特性及TOF检测方法研究[J]. 电子测量与仪器学报, 2017, 31 (7): 1129- 1136
CHEN Zhenhua, ZHENG Zhiyuan, LU Chao Study on ultrasonic propagation characteristics and TOFD detection method in stainless steel weld[J]. Journal of Electronic Measurement and Instrumentation, 2017, 31 (7): 1129- 1136
4 WANG J M, QIAO J P, GUO M C Research on bearing surface defect detection system based on machine vision[J]. Journal of Physics: Conference Series, 2022, 2290 (1): 012061
doi: 10.1088/1742-6596/2290/1/012061
5 LU M H, CHEN C L Detection and classification of bearing surface defects based on machine vision[J]. Applied Sciences, 2021, 11 (4): 1825
doi: 10.3390/app11041825
6 陈昊, 张奔, 黎明, 等 基于图像光流的轴承滚子表面缺陷检测[J]. 仪器仪表学报, 2018, 39 (6): 198- 206
CHEN Hao, ZHANG Ben, LI Ming, et al Bearing roller surface defect detection based on image optical flow[J]. Chinese Journal of Scientific Instrument, 2018, 39 (6): 198- 206
7 陈丹阳, 曹丽, 林一高, 等 轴承圆锥滚子外观缺陷检测研究[J]. 机电工程, 2015, 32 (8): 1084- 1087
CHEN Danyang, CAO Li, LIN Yigao, et al Study on appearance defect detection of bearing tapered roller[J]. Mechanical and Electrical Engineering, 2015, 32 (8): 1084- 1087
8 GIRSHICK R. Fast r-CNN [C]//2015 IEEE International Conference on Computer Vision. Santiago: IEEE, 2015: 1440-1448.
9 袁天乐, 袁巨龙, 朱勇建, 等 基于改进YOLOv5的推力球轴承表面缺陷检测算法[J]. 浙江大学学报: 工学版, 2022, 56 (12): 2349- 2357
YUAN Tianle, YUAN Julong, ZHU Yongjian, et al Surface defect detection algorithm for thrust ball bearing based on improved YOLOv5[J]. Journal of Zhejiang University: Engineering Science, 2022, 56 (12): 2349- 2357
10 曾耀, 高法钦 基于改进YOLOv5的电子元件表面缺陷检测算法[J]. 浙江大学学报: 工学版, 2023, 57 (3): 455- 465
ZENG Yao, GAO Faqin Surface defect detection algorithm for electronic components based on improved YOLOv5[J]. Journal of Zhejiang University: Engineering Science, 2023, 57 (3): 455- 465
11 TAN M, PANG R, LE Q V. Efficientdet: scalable and efficient object detection [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 10781-10790.
12 CONG P C, FENG H, LV K F, et al MYOLO: a light weight fresh Shiitake mushroom detection model based on YOLOv3[J]. Agriculture, 2023, 13 (2): 392
doi: 10.3390/agriculture13020392
13 LIANG C, YAN Z G, REN M, et al Improved YOLOv5 infrared tank target detection method under ground background[J]. Scientific Reports, 2023, 13 (1): 6269
doi: 10.1038/s41598-023-33552-x
14 WANG C Y, BOCHKOVSKIY A, LIAO H Y M. YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2023: 7464–7475.
15 WANG C Y, BOCHKOVSKIY A, LIAO H Y M. Scaled-yolov4: scaling cross stage partial network [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. [S. l. ]: IEEE, 2021: 13029-13038.
16 GE Z, LIU S, WANG F, et al. Yolox: exceeding yolo series in 2021 [EB/OL]. [2023-05-17]. https://arxiv.org/pdf/2107.08430.pdf.
17 SUNKARA R, LUO T. No more strided convolutions or pooling: a new CNN building block for low-resolution images and small objects [C]//European Conference on Computer Vision. [S. l. ]: Springer, 2022: 19–23.
18 JIANG Y, TAN Z, WANG J, et al. GiraffeDet: a heavy-neck paradigm for object detection[EB/OL]. [2023-05-17]. https://arxiv.org/pdf/2202.04256.pdf.
19 LIU S, QI L, QIN H, et al. Path aggregation network for instance segmentation [C] //Proceedings of the IEEE Conference onComputer Vision and Pattern Recognition. Lake City: IEEE, 2018: 8759-8768.
20 TONG Z, CHEN Y, XU Z, et al. Wise-IoU: bounding box regression loss with dynamic focusing mechanism [EB/OL]. [2023-05-17]. https://arxiv.org/pdf/2301.10051.pdf.
21 LIU W, ANGUELOV D, ERHAN D, et al. SSD: singleshot multibox detector [EB/OL]. [2023-05-17]. https://arxiv.org/ pdf/1512.02325.pdf.
22 REDMON J, FARHADI A. YOLOv3: an incremental improvement [EB/OL]. [2023-05-17]. https://arxiv.org/pdf/1804.02767.pdf.
[1] 艾青林,崔景瑞,吕冰海,童桐. 基于损伤区域融合变换的轴承鼓形滚子表面损伤检测方法[J]. 浙江大学学报(工学版), 2023, 57(5): 1009-1020.
[2] 卞佰成,陈田,吴入军,刘军. 基于改进YOLOv3的印刷电路板缺陷检测算法[J]. 浙江大学学报(工学版), 2023, 57(4): 735-743.
[3] 兰欢,余建波. 基于深度学习三维成型的钢板表面缺陷检测[J]. 浙江大学学报(工学版), 2023, 57(3): 466-476.
[4] 袁天乐,袁巨龙,朱勇建,郑翰辰. 基于改进YOLOv5的推力球轴承表面缺陷检测算法[J]. 浙江大学学报(工学版), 2022, 56(12): 2349-2357.
[5] 赵爽,吴君涛,邱欣晨,王奎华,涂园. 基于水平低应变法的高承台桩缺陷检测研究[J]. 浙江大学学报(工学版), 2021, 55(10): 1867-1876.
[6] 陈杨波,伊国栋,张树有. 基于点云特征对比的曲面翘曲变形检测方法[J]. 浙江大学学报(工学版), 2021, 55(1): 81-88.
[7] 叶刚,李毅波,马逐曦,成杰. 基于ViBe的端到端铝带表面缺陷检测识别方法[J]. 浙江大学学报(工学版), 2020, 54(10): 1906-1914.
[8] 秦钟伟,陈捷,洪荣晶,吴伟伟. 摩擦片表面缺陷的视觉显著性检测算法[J]. 浙江大学学报(工学版), 2019, 53(10): 1883-1891.