Please wait a minute...
Journal of ZheJiang University (Engineering Science)  2023, Vol. 57 Issue (9): 1903-1914    DOI: 10.3785/j.issn.1008-973X.2023.09.022
    
On-line quantitative detection of welding arc shape based on object detection
Wei-long HE1(),Ping WANG1,2,*(),Ai-hua ZHANG1,2,Ting-ting LIANG1,Qiang-jie MA1
1. College of Electrical and Information Engineering, Lanzhou University of Technology, Lanzhou 730050, China
2. Key Laboratory of Gansu Advanced Control for Industrial Processes, Lanzhou 730050, China
Download: HTML     PDF(2414KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

A lightweight object detection network YOLO-SEC based on YOLOv5s was proposed to resolve the problem that the arc shape in ultra-narrow gap welding process was difficult to detect online and describe quantitatively. Three lightweight convolutional modules, namely AD-Shufflenet, LC3 and Conv-Maxpool, were designed and applied to the backbone network and neck network of baseline network. Single detection layer inference structure was used in the detection head network, so that the network complexity was effectively reduced. The k-means+ algorithm was designed to cluster labeled anchor frames in the data set and obtain the preset anchor frame size required for network training, and the EIoU positioning loss function was introduced to improve the accuracy of the network. The monocular linear camera calibration algorithm was introduced into the prediction frame drawing function, so that the pixel length could be converted into the actual length by calibration before welding, and the quantitative description of arc size was realized. Experimental results showed that the detection accuracy of YOLO-SEC was 99.2%, the measurement error was less than 0.5%, the reasoning speed was 3.1 ms, and the network volume was 0.6 MB. All the above indexes were better than the YOLOv5s network and the same type of lightweight network.



Key wordsultra-narrow gap welding      arc shape      object detection      camera calibration      lightweight     
Received: 10 September 2022      Published: 16 October 2023
CLC:  TP 391.4  
Fund:  国家自然科学基金资助项目(61866021,62001198,62173170);流程工业综合自动化国家重点实验室联合开放基金资助项目(2021-KF-21-04);甘肃省青年科技基金资助项目(20JR10RA186)
Corresponding Authors: Ping WANG     E-mail: heweilongd@163.com;pingwangsky@163.com
Cite this article:

Wei-long HE,Ping WANG,Ai-hua ZHANG,Ting-ting LIANG,Qiang-jie MA. On-line quantitative detection of welding arc shape based on object detection. Journal of ZheJiang University (Engineering Science), 2023, 57(9): 1903-1914.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2023.09.022     OR     https://www.zjujournals.com/eng/Y2023/V57/I9/1903


基于目标检测的焊接电弧形态在线定量检测

超窄间隙焊接过程中电弧形态难以在线监测和定量描述,为此提出基于YOLOv5s的轻量化目标检测网络YOLO-SEC. 设计3种轻量化卷积模块AD-Shufflenet、LC3和Conv-Maxpool,应用于基线网络的骨干网络和颈网络;在检测头网络中使用单检测层推理结构,有效降低网络复杂度. 设计并使用k-means+算法聚类数据集的已标注锚框,得到网络训练所需的预设锚框尺寸. 引入EIoU定位损失函数,提升网络的准确率. 将单目线性相机标定算法引入预测框绘制函数,在焊接前通过标定将像素长度转化为实际长度,实现电弧尺寸的定量描述. 实验结果表明,YOLO-SEC的检测精度达到99.2%、测量误差小于0.5%、推理速度为3.1 ms、网络体积为0.6 MB,以上指标均优于YOLOv5s网络和同类型轻量化网络.


关键词: 超窄间隙焊接,  电弧形态,  目标检测,  相机标定,  轻量化 
Fig.1 Process of welding arc shape detection
工况 $ {b_\text{g}} $/mm $ U $/V $ {v_\text{f}}{\text{/(mm}} \cdot {{\text{s}}^{ - 1}}) $ $ v{\text{/(mm}} \cdot {{\text{s}}^{ - 1}}) $
1 5.0 22.5 57.1 8.0
2 5.0 23.0 52.6 8.0
Tab.1 Process parameters of welding experiment
Fig.2 Diagram of welding arc image acquisition system
Fig.3 Typical welding arc images in LUT-Arc data set
图片类型 N
训练集 验证集
约束良好 426 109
约束良好(含火焰) 154 38
约束不佳 333 79
约束不佳(含飞溅) 83 23
总计 996 249
Tab.2 LUT-Arc data set welding arc type composition
Fig.4 Structure diagram of baseline network YOLOv5s
Fig.5 Network structure of C3 module and LC3 module
Fig.6 Network structure of Focus module and Conv-Maxpool module
Fig.7 Network structure of Shufflenet V2 module and AD-Shufflenet V2 module
聚类算法 锚框尺寸 $ F $/% $ R_\text{P} $/%
k-means+遗传算法 [11,28 28,11 20,22] 0.831 1
k-means [11,29 29,12 19,21] 0.828 1
k-means+ [11,27 28,11 20,22] 0.831 1
Tab.3 Comparison of clustering effects for different clustering algorithms
Fig.8 Diagram of anchor frame regression process of EIoU
Fig.9 Test results of effects for different variables on performance of calibration method
Fig.10 Actual calibration images of different reference objects
测量对象 $ l $/mm $l_{\rm{m} }$/mm E/%
1 2 3 4 5 6 7
螺丝内径 18.00 18.07 18.03 17.89 18.10 18.04 17.92 18.03 0.37
螺丝外径 24.00 24.10 23.93 23.89 24.09 24.04 24.18 23.94 0.39
螺母内径 20.00 19.85 19.91 20.07 20.04 20.11 20.16 19.97 0.46
螺母外径 22.00 22.06 22.07 22.15 21.92 21.96 22.03 22.03 0.30
焊丝直径 1.60 1.61 1.60 1.59 1.58 1.60 1.61 1.60 0.45
均值 0.39
Tab.4 Actual measurement results of different reference objects
Fig.11 Network structure diagram of YOLO-SEC
网络名称 ${N_{\rm{PC}}}$/106 FLOPs/109 V/MB P/% R/% mAP0.5/% t/ms
YOLOv5s 7.26 14.8 14.4 96.9 96.2 97.8 12.9
YOLO-SEC-A 7.03 14.6 14.0 96.4 96.2 97.4 12.4
YOLO-SEC-B 3.73 2.0 1.0 96.6 97.7 98.0 4.6
YOLO-SEC-C 3.13 1.9 0.8 95.8 97.3 98.1 4.2
YOLO-SEC-D 2.15 1.1 0.6 96.1 95.0 97.7 3.8
YOLO-SEC-E 2.15 1.1 0.7 96.1 96.2 98.1 4.2
YOLO-SEC-F 1.60 0.6 0.4 97.7 98.8 98.9 3.9
YOLO-SEC 1.60 0.6 0.4 97.7 98.5 99.2 3.1
Tab.5 YOLO-SEC ablation test index comparison
Fig.12 Precision-recall curves of different welding arc detection networks
Fig.13 Comparison of comprehensive indicators for YOLO-SEC and four lightweight networks
网络名称 ${N_{{\rm{PC}}} }$/106 FLOPs/109 V/MB P/% R/% mAP0.5/% t/ms
SDD 23.75 59.5 90.7 94.5 95.20 95.85 25.0
YOLOv3 61.94 156.3 18.8 94.8 98.50 98.90 125.0
YOLOv4 64.36 60.50 277.7 85.6 91.57 91.70 27.8
YOLOv3-tiny 8.67 13.3 17.4 97.4 98.10 97.60 14.7
YOLOv4-tiny 6.95 6.1 23.0 95.9 98.33 97.70 14.5
YOLOv5 lite 0.80 8.7 2.1 93.4 95.50 95.90 9.8
YOLOv5 Nano 1.86 4.5 3.8 97.3 96.10 98.40 6.1
YOLO-SEC 1.60 0.6 0.4 97.7 98.50 99.20 3.1
Tab.6 Comparison of test index for different object detection networks
Fig.14 Comparison of detection effect for YOLO-SEC and four lightweight networks
Fig.15 Display interface of upper computer terminal
Fig.16 Actual detection effect of four types of welding arcs
[1]   朱志明, 程世佳, 符平坡, 等 焊接电弧行为特征和稳定性评价研究与进展[J]. 焊接, 2018, (11): 1- 6
ZHU Zhi-ming, CHENG Shi-jia, FU Ping-po, et al Research and progress on behavior characteristics and stability evaluation of welding arc[J]. Welding and Joining, 2018, (11): 1- 6
[2]   MIAO J, LI Y, DONG Z, et al Research progress of low spatter CO2 welding technology [J]. The International Journal of Advanced Manufacturing Technology, 2022, 123: 3715- 3736
doi: 10.1007/s00170-022-10504-9
[3]   陈欢, 朱亮, 姚仁, 等 超窄间隙焊接中送进式焊剂带约束电弧的加热特性[J]. 电焊机, 2021, 51 (9): 41- 45
CHEN Huan, ZHU Liang, YAO Ren, et al Heating characteristics of continuous feeding flux strips constrained arc in ultra-narrow gap welding[J]. Electric Welding Machine, 2021, 51 (9): 41- 45
[4]   朱亮, 张旭磊, 郑韶先, 等 焊剂带约束电弧超窄间隙焊接的实现[J]. 兰州理工大学学报, 2007, 33 (3): 27- 30
ZHU Liang, ZHANG Xu-lei, ZHENG Shao-xian, et al Realization of ultra-narrow gap welding with flux strips constraining arc[J]. Journal of Lanzhou University of Technology, 2007, 33 (3): 27- 30
[5]   龚练, 朱亮, 张爱华 超窄间隙焊接中焊剂片与电弧相 互作用机制分析[J]. 焊接学报, 2017, 38 (12): 57- 60
GONG Lian, ZHU Liang, ZHANG Ai-hua, et al Interaction mechanism of flux band and arc in ultra-narrow gap welding[J]. Transactions of the China Welding Institution, 2017, 38 (12): 57- 60
[6]   任乐, 张爱华, 常东东 基于电弧声的超窄间隙焊接熔合状态识别[J]. 电焊机, 2021, 51 (7): 11- 16
REN Le, ZHANG Ai-hua, CHANG Dong-dong, et al Fusion state recognition of ultra-narrow gap welding based on arc sound[J]. Electric Welding Machine, 2021, 51 (7): 11- 16
[7]   YE D, HUA X, ZHANG J, et al Analysis of arc interference and welding stability in twin wire GMA welding[J]. The International Journal of Advanced Manufacturing Technology, 2015, 81: 627- 633
doi: 10.1007/s00170-015-7225-9
[8]   ZHU Y, WANG Z, LIU R, et al Study on arc behavior and droplet transfer in twin-electrode TIG-MIG indirect arc welding[J]. The International Journal of Advanced Manufacturing Technology, 2022, 120: 6821- 6831
doi: 10.1007/s00170-022-09131-1
[9]   LI W, JI Y, WU J, et al A modified welding image feature extraction algorithm for rotating arc narrow gap MAG welding[J]. Industrial Robot, 2015, 42 (3): 222- 227
doi: 10.1108/IR-11-2014-0407
[10]   王聪, 马国红, 姜芙蓉, 等 基于数字形态学的DE-GMAW电弧图像处理[J]. 热加工工艺, 2012, 41 (19): 145- 147
WANG Cong, MA Guo-hong, JIANG Fu-rong, et al DE-GMAW arc image processing based on mathematical morphology[J]. Hot Working Technology, 2012, 41 (19): 145- 147
[11]   JAMROZIK W, GÓRKA J Assessing MMA welding process stability using machine vision-based arc features tracking system[J]. Sensors, 2021, 21 (1): 84
[12]   程世佳, 朱志明, 符平坡 基于电弧图像的脉冲TIG焊电弧形态及特征温度演变规律[J]. 清华大学学报: 自然科学版, 2021, 61 (9): 994- 1001
CHENG Shi-jia, ZHU Zhi-ming, FU Ping-po Arc shape variations and characteristic temperatures of pulsed TIG welding arcs based on observed arc images[J]. Journal of Tsinghua University: Science and Technology, 2021, 61 (9): 994- 1001
[13]   ZHU C, TANG X, HE Y, et al Study on arc characteristics and their influences on weld bead geometry in narrow gap GMAW of 5083 Al-alloy[J]. The International Journal of Advanced Manufacturing Technology, 2017, 90: 2513- 2525
doi: 10.1007/s00170-016-9563-7
[14]   REN S, HE K, GIRSHICK R, et al Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39 (6): 1137- 1149
doi: 10.1109/TPAMI.2016.2577031
[15]   LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot MultiBox detector [C]// Proceedings of European Conference on Computer Vision. Amsterdam: Springer, 2016: 21–37.
[16]   REDMON J, DIVVAL S, GIRSHICK R, et al. You only look once: unified, real-time object detection [C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 779–788.
[17]   REDMON J, FARHADI A. YOLO9000: better, faster, stronger [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 6517–6525.
[18]   REDMON J, FARHADI A. YOLOv3: an incremental improvement [EB/OL]. (2018-01-20)[2022-08-12]. https://arxiv.org/pdf/1804.02767v1.pdf.
[19]   BOCHKOVSKIY A, WANG C Y, LIAO H Y M. Yolov4: optimal speed and accuracy of object detection [EB/OL]. (2020-04-23) [2022-08-12] https://arxiv.org/pdf/2004.10934.pdf.
[20]   郝琨, 王阔, 王贝贝 基于改进Mobilenet-YOLOv3的轻量级水下生物检测算法[J]. 浙江大学学报: 工学版, 2022, 56 (8): 1622- 1632
HAO Kun, WANG Kuo, WANG Bei-bei Lightweight underwater biological detection algorithm based on improved Mobilenet-YOLOv3[J]. Journal of Zhejiang University: Engineering Science, 2022, 56 (8): 1622- 1632
[21]   董红召, 方浩杰, 张楠 旋转框定位的多尺度再生物品目标检测算法[J]. 浙江大学学报: 工学版, 2022, 56 (1): 16- 25
DONG Hong-zhao, FANG Hao-jie, ZHANG Nan Multi-scale object detection algorithm for recycled objects based on rotating block positioning[J]. Journal of Zhejiang University: Engineering Science, 2022, 56 (1): 16- 25
[22]   ZHANG Y F, REN W, ZHANG Z, et al Focal and efficient IOU loss for accurate bounding box regression[J]. Neurocomputing, 2022, 506: 146- 157
doi: 10.1016/j.neucom.2022.07.042
[23]   LIU S, QI L, QIN H, et al. Path aggregation network for instance segmentation [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 8759-8768.
[24]   WANG C Y, LIAO H Y M, WU Y H, et al. CSPNet: a new backbone that can enhance learning capability of CNN [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Seattle: IEEE, 2020: 390-391.
[25]   MA N, ZHANG X, ZHENG H T, et al. Shufflenet V2: practical guidelines for efficient CNN architecture design [C]// Proceedings of the European Conference on Computer Vision. [S.l.]: Springer, 2018: 122-138.
[26]   ZHENG Z, WANG P, LIU W, et al. Distance-IoU loss: faster and better learning for bounding box regression [C]// Proceedings of the AAAI Conference on Artificial Intelligence. 2020, 34(7): 12993-13000.
[27]   尹英杰, 徐德, 张正涛 等 基于单目视觉的平面测量[J]. 电子测量与仪器学报, 2013, 27 (4): 347- 352
YIN Ying-jie, XU De, ZHANG Zheng-tao, et al Plane measurement based on monocular vision[J]. Journal of Electronic Measurement and Instrument, 2013, 27 (4): 347- 352
[1] Xin JIN,Jian-jun ZHUANG,Zi-heng XU. Lightweight YOLOv5s network-based algorithm for identifying hazardous objects under vehicles[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(8): 1516-1526.
[2] Hao-ran GUO,Ji-chang GUO,Yu-dong WANG. Lightweight semantic segmentation network for underwater image[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(7): 1278-1286.
[3] Yan ZHANG,Jing-xue SUN,Ye-mei SUN,Shu-dong LIU,Chuan-qi WANG. Lightweight object detection based on split attention and linear transformation[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(6): 1195-1204.
[4] Yao ZENG,Fa-qin GAO. Surface defect detection algorithm of electronic components based on improved YOLOv5[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(3): 455-465.
[5] Kun HAO,Kuo WANG,Bei-bei WANG. Lightweight underwater biological detection algorithm based on improved Mobilenet-YOLOv3[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(8): 1622-1632.
[6] Na ZHANG,Xu-lei QI,Xiao-an BAO,Biao WU,Xiao-mei TU,Yu-ting JIN. Single-stage object detection algorithm based on optimizing position prediction[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(4): 783-794.
[7] Jing-hui CHU,Li-dong SHI,Pei-guang JING,Wei LV. Context-aware knowledge distillation network for object detection[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(3): 503-509.
[8] Nan-jing YU,Xiao-biao FAN,Tian-min DENG,Guo-tao MAO. Ship detection algorithm in complex backgrounds via multi-head self-attention[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(12): 2392-2402.
[9] Yu XIE,Zi-qun BAO,Na ZHANG,Biao WU,Xiao-mei TU,Xiao-an BAO. Object detection algorithm based on feature enhancement and deep fusion[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(12): 2403-2415.
[10] Fei LI,Kun HU,Yong ZHANG,Wen-shan WANG,Hao JIANG. Multi-dimensional detection of longitudinal tearing of conveyor belt based on YOLOv4 of hybrid domain attention[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(11): 2156-2167.
[11] Rong ZHANG,Wei ZHANG. Fire detection algorithm based on improved GhostNet-FCOS[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(10): 1891-1899.
[12] Kai DU,Guo-rong ZHU,Jiang-hua LU,Mu-ye PANG. Metal object detection method in wireless electric vehicle charging system[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(1): 56-62.
[13] Jiang-bo XU,Yuan-zhi WANG,Yu QI,Bao-hua CAO,Yong-zhen LUO,Chang-gen YAN,Xiao-hua YANG,Han BAO,Yu-zhou XIANG. Deformation characteristics of fiber-reinforced foam lightweight soil under cyclic loading and unloading[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(1): 111-117.
[14] Ying-jie NIU,Yan-chen SU,Dun-cheng CHENG,Jia LIAO,Hai-bo ZHAO,Yong-qiang GAO. High-speed rail contact network U-holding nut fault detection algorithm[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(10): 1912-1921.
[15] Wen-bin XIN,Hui-min HAO,Ming-long BU,Yuan LAN,Jia-hai HUANG,Xiao-yan XIONG. Static gesture real-time recognition method based on ShuffleNetv2-YOLOv3 model[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(10): 1815-1824.