1. College of Mechanical Engineering, Zhejiang University of Technology, Hangzhou 310023, China 2. College of Mechanical Engineering, Zhejiang University of Science and Technology, Hangzhou 310023, China
An automatic extraction detection area preprocessing and a multi-head self-attention mechanism module in the improved Transformer were proposed, in order to improve the accuracy and recall rate of the surface defect detection of thrust ball bearings, and enhance the anti-interference ability of the model. The proposed module was introduced into the feature network ignoring irrelevant noise information and focusing on the key information, and the extraction ability of small and medium-sized surface defects was improved. Instance normalization was used instead of Batch normalization to improve the convergence speed and detection accuracy during model training. Results show that in the thrust ball bearing surface defect detection dataset, the accuracy rate of the improved YOLOv5 model reaches 87.0%, the recall rate reaches 83.0%, the average precision reaches 86.1%, and the average detection time per image was 14.96 ms. Compared with the YOLOv5s model, the accuracy of the improved model is increased by 1.5%, the recall rate is increased by 7.3%, and the average precision is increased by 7.9%. Compared with the original model, the improved YOLOv5 model has better defect positioning ability and higher accuracy, and can reduce interference of foreign objects in the detection process on detection results. A detection speed of the improved YOLOv5 model can meet the requirements of industrial mass detection.
Fig.2Flow chart of preprocessing for automatic extraction of target area from thrust ball bearing images
Fig.3Vision Transformer network structure
Fig.4Improvement of Transformer multi-head self-attention mechanism module
Fig.5Improve YOLOv5 model structure
Fig.6Thrust ball bearing surface defect sample
Fig.7Thrust ball bearing image data enhancement
模型
AP
p
r
mAP@0.5
t/ms
划伤
压印
缺球
YOLOv5s
0.567
0.789
0.988
0.855
0.757
0.782
8.28
YOLOv5m
0.550
0.807
0.991
0.811
0.783
0.783
15.64
Tab.1Comparison of YOLOv5s and YOLOv5m detection result of bearing
预处理
AP
p
r
mAP@0.5
t/ms
划伤
压印
缺球
无
0.567
0.789
0.988
0.855
0.757
0.782
8.28
有
0.676
0.846
0.986
0.826
0.816
0.836
8.89
Tab.2Comparison of test results before and after pretreatment of bearing
Fig.8Comparison of loss convergence before and after improvement
算法
预处理
AP
p
r
mAP@0.5
划伤
压印
缺球
BN
无
0.567
0.789
0.988
0.855
0.757
0.782
IN
无
0.592
0.835
0.994
0.840
0.791
0.807
BN
有
0.676
0.846
0.986
0.826
0.816
0.836
IN
有
0.684
0.855
0.993
0.862
0.792
0.844
Tab.3Improved comparison of detection results for normalization functions
Fig.9Model base network structure
实验
模型
算法
AP
F1
mAP@0.5
划伤
压印
缺球
1
原
BN
0.676
0.846
0.986
0.821
0.836
2
图5(a)
BN
0.666
0.864
0.993
0.821
0.841
3
图5(b)
BN
0.698
0.851
0.988
0.831
0.846
4
图5(c)
BN
0.566
0.869
0.987
0.814
0.807
5
图5(d)
BN
0.714
0.869
0.989
0.832
0.857
6
图5(b)
IN
0.715
0.869
0.992
0.845
0.859
7
图5(d)
IN
0.717
0.876
0.991
0.850
0.861
Tab.4Add multi-head self-attention mechanism module detection result comparison
Fig.10Comparison of detection results of surface defects on bearings before and after YOLOv5s improvement
模型
AP
mAP@0.5
t/ms
划伤
压印
缺球
YOLOv3
0.586
0.813
0.981
0.806
34.39
YOLOv3-SPP
0.65
0.815
0.991
0.819
30.73
Faster-RCNN
0.468
0.690
0.984
0.714
124
YOLOv5s
0.676
0.846
0.986
0.836
8.28
本研究
0.717
0.876
0.991
0.861
14.96
Tab.5Comparison of proposed model detection results with mainstream models
[1]
刘桥方, 严枫 我国轴承制造技术的现状及其发展趋势[J]. 轴承, 2005, (6): 42- 45 LIU Qiao-fang, YAN Feng Current situation and development trend of Chinese bearing manufacturing technology[J]. Bearing, 2005, (6): 42- 45
doi: 10.3969/j.issn.1000-3762.2005.06.018
[2]
ZOU Y, DU D, CHANG B, et al Automatic weld defect detection method based on Kalman filtering for real-time radiographic inspection of spiral pipe[J]. NDT&E International, 2015, 72: 1- 9
[3]
BEARD M D, LOWE M J S Non-destructive testing of rock bolts using guided ultrasonic waves[J]. International Journal of Rock Mechanics and Mining Sciences, 2003, 40 (4): 527- 536
doi: 10.1016/S1365-1609(03)00027-3
[4]
TITMAN D J Applications of thermography in non-destructive testing of structures[J]. NDT&E International, 2001, 34 (2): 149- 154
[5]
LU Q Y, WONG C H Applications of non-destructive testing techniques for post-process control of additively manufactured parts[J]. Virtual and Physical Prototyping, 2017, 12 (4): 301- 321
doi: 10.1080/17452759.2017.1357319
[6]
GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation [C]// 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus: IEEE, 2014: 580-587.
[7]
HE K, ZHANG X, REN S, et al Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37 (9): 1904- 1916
doi: 10.1109/TPAMI.2015.2389824
[8]
GIRSHICK R. Fast R-CNN [J]. 2015 IEEE International Conference on Computer Vision. Santiago: IEEE, 2015: 1440-1448.
[9]
REN S, HE K, GIRSHICK R, et al Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39 (6): 1137- 1149
doi: 10.1109/TPAMI.2016.2577031
[10]
DAI J, LI Y, HE K, et al. R-FCN: object detection via region-based fully convolutional networks [C]// Proceedings of the 30th International Conference on Neural Information Processing Systems. Barcelona: [s. n.], 2016: 379-387.
[11]
HE K, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN [C]// 2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 2961-2969.
[12]
REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection [C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 779-788.
[13]
REDMON J, FARHADI A. YOLO9000: better, faster, stronger [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 7263-7271.
[14]
REDMON J, FARHADI A. YOLOv3: an incremental improvement [EB/OL]. [2021-10-08]. https://arxiv.org/pdf/1804.02767.pdf.
[15]
BOCHKOVSKIY A, WANG C Y, LIAO H Y M. YOLOv4: optimal speed and accuracy of object detection [EB/OL]. [2021-10-08]. https://arxiv.org/pdf/2004.10934v1.pdf.
[16]
LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot multiBox detector [C]// European Conference on Computer Vision 2016. [S. l.]: Springer, 2016: 21-37.
[17]
FU C Y, LIU W, RANGA A, et al. DSSD: deconvolutional single shot detector [EB/OL]. [2021-10-08]. https://arxiv.org/pdf/1701.06659.pdf .
[18]
LI Z, ZHOU F. FSSD: feature fusion single shot multibox detector [EB/OL]. [2021-10-08]. https://arxiv.org/pdf/1712.00960.pdf.
[19]
LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection [C]// 2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 2980-2988.
[20]
王鹏. 基于机器视觉的钢球表面缺陷检测系统[D]. 哈尔滨: 哈尔滨理工大学, 2005. WANG Peng. Surface defect detect system of steel ball based on machine vision [D]. Harbin: Harbin University of Science and Technology, 2005.
[21]
陈琦. 基于机器视觉的滑动轴承内表面缺陷自动检测系统的研究[D]. 镇江: 江苏大学, 2017. CHEN Qi. Research on automatic detection system of inner surface defects of sliding bearing based on machine vision [D]. Zhenjiang: Jiangsu University, 2017.
[22]
KUNAKORNVONG P, TANGKONGKIET C, SOORAKSA P. Defect detection on air bearing surface with gray level co-occurrence matrix [C]// The 4th Joint International Conference on Information and Communication Technology, Electronic and Electrical Engineering. Chiang Rai: IEEE, 2014: 1-4.
[23]
DENG S, CAI W, XU Q, et al. Defect detection of bearing surfaces based on machine vision technique [C]// 2010 International Conference on Computer Application and System Modeling. Taiyuan: IEEE, 2010: V4−548−V4−554.
[24]
李维刚, 叶欣, 赵云涛, 等 基于改进YOLOv3算法的带钢表面缺陷检测[J]. 电子学报, 2020, (7): 1284- 1292 LI Wei-gang, YE Xin, ZHAO Yun-tao, et al Strip steel surface defect detection based on improved YOLOv3 algorithm[J]. Acta Electronica Sinica, 2020, (7): 1284- 1292
doi: 10.3969/j.issn.0372-2112.2020.07.006
[25]
张磊, 郎贤礼, 王乐 基于图像融合与YOLOv3的铝型材表面缺陷检测[J]. 计算机与现代化, 2020, (11): 8- 15 ZHANG Lei, LANG Xian-li, WANG Le Surface defect detection of aluminum profiles based on image fusion and YOLOv3[J]. Computer and Modernization, 2020, (11): 8- 15
doi: 10.3969/j.issn.1006-2475.2020.11.002
[26]
李浪怡, 刘强, 邹一鸣, 等 基于改进YOLOv5算法的轨面缺陷检测[J]. 五邑大学学报: 自然科学版, 2021, 35 (3): 43- 48 LI Lang-yi, LIU Qiang, ZOU Yi-ming, et al Rail surface defect detection based on improved YOLOv5 algorithm[J]. Journal of Wuyi University: Natural Science Edition, 2021, 35 (3): 43- 48
[27]
JIN R, NIU Q Automatic fabric defect detection based on an improved YOLOv5[J]. Mathematical Problems in Engineering, 2021, 7321394
[28]
ZHAO Z, YANG X, ZHOU Y, et al Real-time detection of particleboard surface defects based on improved YOLOV5 target detection[J]. Scientific Reports, 2021, 11: 21777
doi: 10.1038/s41598-021-01084-x
[29]
IOFFE S, SZEGEDY C. Batch normalization: accelerating deep network training by reducing internal covariate shift [C]// Proceedings of the 32nd International conference on machine learning. Lille: [s. n.], 2015: 448-456.
[30]
BA J L, KIROS J R, HINTON G E. Layer normalization [EB/OL]. [2021-10-08]. https://arxiv.org/pdf/1607.06450.pdf.
[31]
LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 2117-2125.
[32]
LIU S, QI L, QIN H, et al. Path aggregation network for instance segmentation [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Lake City: IEEE, 2018: 8759-8768.
