Please wait a minute...
浙江大学学报(工学版)
Journal of ZheJiang University (Engineering Science)  2022, Vol. 56 Issue (12): 2349-2357    DOI: 10.3785/j.issn.1008-973X.2022.12.004
    
Surface defect detection algorithm of thrust ball bearing based on improved YOLOv5
Tian-le YUAN1(),Ju-long YUAN1,*(),Yong-jian ZHU2,Han-chen ZHENG1
1. College of Mechanical Engineering, Zhejiang University of Technology, Hangzhou 310023, China
2. College of Mechanical Engineering, Zhejiang University of Science and Technology, Hangzhou 310023, China
Download: HTML     PDF(1216KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

An automatic extraction detection area preprocessing and a multi-head self-attention mechanism module in the improved Transformer were proposed, in order to improve the accuracy and recall rate of the surface defect detection of thrust ball bearings, and enhance the anti-interference ability of the model. The proposed module was introduced into the feature network ignoring irrelevant noise information and focusing on the key information, and the extraction ability of small and medium-sized surface defects was improved. Instance normalization was used instead of Batch normalization to improve the convergence speed and detection accuracy during model training. Results show that in the thrust ball bearing surface defect detection dataset, the accuracy rate of the improved YOLOv5 model reaches 87.0%, the recall rate reaches 83.0%, the average precision reaches 86.1%, and the average detection time per image was 14.96 ms. Compared with the YOLOv5s model, the accuracy of the improved model is increased by 1.5%, the recall rate is increased by 7.3%, and the average precision is increased by 7.9%. Compared with the original model, the improved YOLOv5 model has better defect positioning ability and higher accuracy, and can reduce interference of foreign objects in the detection process on detection results. A detection speed of the improved YOLOv5 model can meet the requirements of industrial mass detection.

Key words： deep learning      thrust ball bearing      YOLOV5      Transformer      attention mechanism      surface defect detection.     
Received: 10 December 2021      Published: 03 January 2023
CLC:  TP 391.41  
Fund:  国家重点研发计划资助项目(2018YFB2000402)；国家自然科学基金资助项目(U1809221)；浙江省基础公益计划资助项目(LGG21E050006)
Corresponding Authors: Ju-long YUAN     E-mail: 435282558@qq.com;jlyuan@zjut.edu.cn
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Tian-le YUAN
Ju-long YUAN
Yong-jian ZHU
Han-chen ZHENG
Cite this article:

Tian-le YUAN,Ju-long YUAN,Yong-jian ZHU,Han-chen ZHENG. Surface defect detection algorithm of thrust ball bearing based on improved YOLOv5. Journal of ZheJiang University (Engineering Science), 2022, 56(12): 2349-2357.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2022.12.004     OR     https://www.zjujournals.com/eng/Y2022/V56/I12/2349


基于改进YOLOv5的推力球轴承表面缺陷检测算法

为了提高推力球轴承表面缺陷检测的精确率和召回率，增强模型抗干扰能力，提出自动提取检测区域预处理和改进Transformer中的多头自注意力机制模块. 在特征网络引入所提模块，忽略无关噪声信息而关注重点信息，提升中小表面缺陷的提取能力. 使用实例归一化代替批量归一化，提高模型训练时的收敛速度和检测精度. 结果表明，在推力球轴承表面缺陷检测数据集中，改进YOLOv5模型的准确率达到87.0%，召回率达到83.0%，平均精度达到86.1%，平均每张图片检测时间为14.96 ms. 相比于YOLOv5s模型，改进模型的准确率提升1.5%，召回率提升7.3%，平均精度提升7.9%. 与原模型相比，改进YOLOv5模型有更好的缺陷定位能力和较高的准确度，能够减小检测过程中的异物对检测结果造成的干扰，检测速度满足工业大批量检测的要求.


关键词： 深度学习,  推力球轴承,  YOLOv5,  Transformer,  注意力机制,  表面缺陷检测 
Fig.1 YOLOv5 network structure
Fig.2 Flow chart of preprocessing for automatic extraction of target area from thrust ball bearing images
Fig.3 Vision Transformer network structure
Fig.4 Improvement of Transformer multi-head self-attention mechanism module
Fig.5 Improve YOLOv5 model structure
Fig.6 Thrust ball bearing surface defect sample
Fig.7 Thrust ball bearing image data enhancement
模型 AP p r mAP@0.5 t/ms
划伤 压印 缺球
YOLOv5s 0.567 0.789 0.988 0.855 0.757 0.782 8.28
YOLOv5m 0.550 0.807 0.991 0.811 0.783 0.783 15.64
Tab.1 Comparison of YOLOv5s and YOLOv5m detection result of bearing
预处理 AP p r mAP@0.5 t/ms
划伤 压印 缺球
无 0.567 0.789 0.988 0.855 0.757 0.782 8.28
有 0.676 0.846 0.986 0.826 0.816 0.836 8.89
Tab.2 Comparison of test results before and after pretreatment of bearing
Fig.8 Comparison of loss convergence before and after improvement
算法 预处理 AP p r mAP@0.5
划伤 压印 缺球
BN 无 0.567 0.789 0.988 0.855 0.757 0.782
IN 无 0.592 0.835 0.994 0.840 0.791 0.807
BN 有 0.676 0.846 0.986 0.826 0.816 0.836
IN 有 0.684 0.855 0.993 0.862 0.792 0.844
Tab.3 Improved comparison of detection results for normalization functions
Fig.9 Model base network structure
实验 模型 算法 AP F1 mAP@0.5
划伤 压印 缺球
1 原 BN 0.676 0.846 0.986 0.821 0.836
2 图5(a) BN 0.666 0.864 0.993 0.821 0.841
3 图5(b) BN 0.698 0.851 0.988 0.831 0.846
4 图5(c) BN 0.566 0.869 0.987 0.814 0.807
5 图5(d) BN 0.714 0.869 0.989 0.832 0.857
6 图5(b) IN 0.715 0.869 0.992 0.845 0.859
7 图5(d) IN 0.717 0.876 0.991 0.850 0.861
Tab.4 Add multi-head self-attention mechanism module detection result comparison
Fig.10 Comparison of detection results of surface defects on bearings before and after YOLOv5s improvement
模型 AP mAP@0.5 t/ms
划伤 压印 缺球
YOLOv3 0.586 0.813 0.981 0.806 34.39
YOLOv3-SPP 0.65 0.815 0.991 0.819 30.73
Faster-RCNN 0.468 0.690 0.984 0.714 124
YOLOv5s 0.676 0.846 0.986 0.836 8.28
本研究 0.717 0.876 0.991 0.861 14.96
Tab.5 Comparison of proposed model detection results with mainstream models
[1]   刘桥方, 严枫 我国轴承制造技术的现状及其发展趋势[J]. 轴承, 2005, (6): 42- 45
LIU Qiao-fang, YAN Feng Current situation and development trend of Chinese bearing manufacturing technology[J]. Bearing, 2005, (6): 42- 45
doi: 10.3969/j.issn.1000-3762.2005.06.018
[2]   ZOU Y, DU D, CHANG B, et al Automatic weld defect detection method based on Kalman filtering for real-time radiographic inspection of spiral pipe[J]. NDT&E International, 2015, 72: 1- 9
[3]   BEARD M D, LOWE M J S Non-destructive testing of rock bolts using guided ultrasonic waves[J]. International Journal of Rock Mechanics and Mining Sciences, 2003, 40 (4): 527- 536
doi: 10.1016/S1365-1609(03)00027-3
[4]   TITMAN D J Applications of thermography in non-destructive testing of structures[J]. NDT&E International, 2001, 34 (2): 149- 154
[5]   LU Q Y, WONG C H Applications of non-destructive testing techniques for post-process control of additively manufactured parts[J]. Virtual and Physical Prototyping, 2017, 12 (4): 301- 321
doi: 10.1080/17452759.2017.1357319
[6]   GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation [C]// 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus: IEEE, 2014: 580-587.
[7]   HE K, ZHANG X, REN S, et al Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37 (9): 1904- 1916
doi: 10.1109/TPAMI.2015.2389824
[8]   GIRSHICK R. Fast R-CNN [J]. 2015 IEEE International Conference on Computer Vision. Santiago: IEEE, 2015: 1440-1448.
[9]   REN S, HE K, GIRSHICK R, et al Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39 (6): 1137- 1149
doi: 10.1109/TPAMI.2016.2577031
[10]   DAI J, LI Y, HE K, et al. R-FCN: object detection via region-based fully convolutional networks [C]// Proceedings of the 30th International Conference on Neural Information Processing Systems. Barcelona: [s. n.], 2016: 379-387.
[11]   HE K, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN [C]// 2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 2961-2969.
[12]   REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection [C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 779-788.
[13]   REDMON J, FARHADI A. YOLO9000: better, faster, stronger [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 7263-7271.
[14]   REDMON J, FARHADI A. YOLOv3: an incremental improvement [EB/OL]. [2021-10-08]. https://arxiv.org/pdf/1804.02767.pdf.
[15]   BOCHKOVSKIY A, WANG C Y, LIAO H Y M. YOLOv4: optimal speed and accuracy of object detection [EB/OL]. [2021-10-08]. https://arxiv.org/pdf/2004.10934v1.pdf.
[16]   LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot multiBox detector [C]// European Conference on Computer Vision 2016. [S. l.]: Springer, 2016: 21-37.
[17]   FU C Y, LIU W, RANGA A, et al. DSSD: deconvolutional single shot detector [EB/OL]. [2021-10-08]. https://arxiv.org/pdf/1701.06659.pdf .
[18]   LI Z, ZHOU F. FSSD: feature fusion single shot multibox detector [EB/OL]. [2021-10-08]. https://arxiv.org/pdf/1712.00960.pdf.
[19]   LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection [C]// 2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 2980-2988.
[20]   王鹏. 基于机器视觉的钢球表面缺陷检测系统[D]. 哈尔滨: 哈尔滨理工大学, 2005.
WANG Peng. Surface defect detect system of steel ball based on machine vision [D]. Harbin: Harbin University of Science and Technology, 2005.
[21]   陈琦. 基于机器视觉的滑动轴承内表面缺陷自动检测系统的研究[D]. 镇江: 江苏大学, 2017.
CHEN Qi. Research on automatic detection system of inner surface defects of sliding bearing based on machine vision [D]. Zhenjiang: Jiangsu University, 2017.
[22]   KUNAKORNVONG P, TANGKONGKIET C, SOORAKSA P. Defect detection on air bearing surface with gray level co-occurrence matrix [C]// The 4th Joint International Conference on Information and Communication Technology, Electronic and Electrical Engineering. Chiang Rai: IEEE, 2014: 1-4.
[23]   DENG S, CAI W, XU Q, et al. Defect detection of bearing surfaces based on machine vision technique [C]// 2010 International Conference on Computer Application and System Modeling. Taiyuan: IEEE, 2010: V4−548−V4−554.
[24]   李维刚, 叶欣, 赵云涛, 等 基于改进YOLOv3算法的带钢表面缺陷检测[J]. 电子学报, 2020, (7): 1284- 1292
LI Wei-gang, YE Xin, ZHAO Yun-tao, et al Strip steel surface defect detection based on improved YOLOv3 algorithm[J]. Acta Electronica Sinica, 2020, (7): 1284- 1292
doi: 10.3969/j.issn.0372-2112.2020.07.006
[25]   张磊, 郎贤礼, 王乐 基于图像融合与YOLOv3的铝型材表面缺陷检测[J]. 计算机与现代化, 2020, (11): 8- 15
ZHANG Lei, LANG Xian-li, WANG Le Surface defect detection of aluminum profiles based on image fusion and YOLOv3[J]. Computer and Modernization, 2020, (11): 8- 15
doi: 10.3969/j.issn.1006-2475.2020.11.002
[26]   李浪怡, 刘强, 邹一鸣, 等 基于改进YOLOv5算法的轨面缺陷检测[J]. 五邑大学学报: 自然科学版, 2021, 35 (3): 43- 48
LI Lang-yi, LIU Qiang, ZOU Yi-ming, et al Rail surface defect detection based on improved YOLOv5 algorithm[J]. Journal of Wuyi University: Natural Science Edition, 2021, 35 (3): 43- 48
[27]   JIN R, NIU Q Automatic fabric defect detection based on an improved YOLOv5[J]. Mathematical Problems in Engineering, 2021, 7321394
[28]   ZHAO Z, YANG X, ZHOU Y, et al Real-time detection of particleboard surface defects based on improved YOLOV5 target detection[J]. Scientific Reports, 2021, 11: 21777
doi: 10.1038/s41598-021-01084-x
[29]   IOFFE S, SZEGEDY C. Batch normalization: accelerating deep network training by reducing internal covariate shift [C]// Proceedings of the 32nd International conference on machine learning. Lille: [s. n.], 2015: 448-456.
[30]   BA J L, KIROS J R, HINTON G E. Layer normalization [EB/OL]. [2021-10-08]. https://arxiv.org/pdf/1607.06450.pdf.
[31]   LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 2117-2125.
[32]   LIU S, QI L, QIN H, et al. Path aggregation network for instance segmentation [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Lake City: IEEE, 2018: 8759-8768.
[1] Jin-zhen LIU,Fei CHEN,Hui XIONG. Open electrical impedance imaging algorithm based on multi-scale residual network model[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(9): 1789-1795.
[2] Wan-liang WANG,Tie-jun WANG,Jia-cheng CHEN,Wen-bo YOU. Medical image segmentation method combining multi-scale and multi-head attention[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(9): 1796-1805.
[3] Kun HAO,Kuo WANG,Bei-bei WANG. Lightweight underwater biological detection algorithm based on improved Mobilenet-YOLOv3[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(8): 1622-1632.
[4] Ren-peng MO,Xiao-sheng SI,Tian-mei LI,Xu ZHU. Bearing life prediction based on multi-scale features and attention mechanism[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(7): 1447-1456.
[5] Yong-sheng ZHAO,Rui-xiang LI,Na-na NIU,Zhi-yong ZHAO. Shape control method of fuselage driven by digital twin[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(7): 1457-1463.
[6] You-wei WANG,Shuang TONG,Li-zhou FENG,Jian-ming ZHU,Yang LI,Fu CHEN. New inductive microblog rumor detection method based on graph convolutional network[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(5): 956-966.
[7] Xiao-chen JU,Xin-xin ZHAO,Sheng-sheng QIAN. Self-attention mechanism based bridge bolt detection algorithm[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(5): 901-908.
[8] Li HE,Shan-min PANG. Face reconstruction from voice based on age-supervised learning and face prior information[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(5): 1006-1016.
[9] Xue-qin ZHANG,Tian-ren LI. Breast cancer pathological image classification based on Cycle-GAN and improved DPN network[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(4): 727-735.
[10] Meng XU,Dan WANG,Zhi-yuan LI,Yuan-fang CHEN. IncepA-EEGNet: P300 signal detection method based on fusion of Inception network and attention mechanism[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(4): 745-753, 782.
[11] Chang-yuan LIU,Xian-ping HE,Xiao-jun BI. Efficient network vehicle recognition combined with attention mechanism[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(4): 775-782.
[12] Qiao-hong CHEN,Hao-lei PEI,Qi SUN. Image caption based on relational reasoning and context gate mechanism[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(3): 542-549.
[13] Jing-hui CHU,Li-dong SHI,Pei-guang JING,Wei LV. Context-aware knowledge distillation network for object detection[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(3): 503-509.
[14] Ruo-ran CHENG,Xiao-li ZHAO,Hao-jun ZHOU,Han-chen YE. Review of Chinese font style transfer research based on deep learning[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(3): 510-519, 530.
[15] Guo-peng ZHANG,Zi-han LI,Hao WANG,zheng ZHENG. Isolated AC-DC solid state transformer front and rear stages integrated sliding mode control[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(3): 622-630.