Please wait a minute...
浙江大学学报(工学版)
浙江大学学报(工学版)  2023, Vol. 57 Issue (9): 1903-1914    DOI: 10.3785/j.issn.1008-973X.2023.09.022
电子、通信与自动控制技术     
基于目标检测的焊接电弧形态在线定量检测
何卫隆1(),王平1,2,*(),张爱华1,2,梁婷婷1,马强杰1
1. 兰州理工大学 电气工程与信息工程学院，甘肃 兰州 730050
2. 甘肃省工业过程先进控制重点实验室，甘肃 兰州 730050
On-line quantitative detection of welding arc shape based on object detection
Wei-long HE1(),Ping WANG1,2,*(),Ai-hua ZHANG1,2,Ting-ting LIANG1,Qiang-jie MA1
1. College of Electrical and Information Engineering, Lanzhou University of Technology, Lanzhou 730050, China
2. Key Laboratory of Gansu Advanced Control for Industrial Processes, Lanzhou 730050, China
 全文: PDF(2414 KB)   HTML
摘要：

超窄间隙焊接过程中电弧形态难以在线监测和定量描述，为此提出基于YOLOv5s的轻量化目标检测网络YOLO-SEC. 设计3种轻量化卷积模块AD-Shufflenet、LC3和Conv-Maxpool，应用于基线网络的骨干网络和颈网络；在检测头网络中使用单检测层推理结构，有效降低网络复杂度. 设计并使用k-means+算法聚类数据集的已标注锚框，得到网络训练所需的预设锚框尺寸. 引入EIoU定位损失函数，提升网络的准确率. 将单目线性相机标定算法引入预测框绘制函数，在焊接前通过标定将像素长度转化为实际长度，实现电弧尺寸的定量描述. 实验结果表明，YOLO-SEC的检测精度达到99.2%、测量误差小于0.5%、推理速度为3.1 ms、网络体积为0.6 MB，以上指标均优于YOLOv5s网络和同类型轻量化网络．
关键词： 超窄间隙焊接;  电弧形态;  目标检测;  相机标定;  轻量化    
Abstract:

A lightweight object detection network YOLO-SEC based on YOLOv5s was proposed to resolve the problem that the arc shape in ultra-narrow gap welding process was difficult to detect online and describe quantitatively. Three lightweight convolutional modules, namely AD-Shufflenet, LC3 and Conv-Maxpool, were designed and applied to the backbone network and neck network of baseline network. Single detection layer inference structure was used in the detection head network, so that the network complexity was effectively reduced. The k-means+ algorithm was designed to cluster labeled anchor frames in the data set and obtain the preset anchor frame size required for network training, and the EIoU positioning loss function was introduced to improve the accuracy of the network. The monocular linear camera calibration algorithm was introduced into the prediction frame drawing function, so that the pixel length could be converted into the actual length by calibration before welding, and the quantitative description of arc size was realized. Experimental results showed that the detection accuracy of YOLO-SEC was 99.2%, the measurement error was less than 0.5%, the reasoning speed was 3.1 ms, and the network volume was 0.6 MB. All the above indexes were better than the YOLOv5s network and the same type of lightweight network.
Key words: ultra-narrow gap welding    arc shape    object detection    camera calibration    lightweight
收稿日期: 2022-09-10 出版日期: 2023-10-16
CLC:  TP 391.4  
基金资助: 国家自然科学基金资助项目(61866021，62001198，62173170）；流程工业综合自动化国家重点实验室联合开放基金资助项目(2021-KF-21-04)；甘肃省青年科技基金资助项目(20JR10RA186)
通讯作者: 王平     E-mail: heweilongd@163.com;pingwangsky@163.com
作者简介: 何卫隆（1998—），男，硕士生，从事智能焊接和机器视觉研究. orcid.org/0000-0002-3132-7220. E-mail： heweilongd@163.com
服务　　
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章　　
何卫隆
王平
张爱华
梁婷婷
马强杰

引用本文:

何卫隆,王平,张爱华,梁婷婷,马强杰. 基于目标检测的焊接电弧形态在线定量检测[J]. 浙江大学学报(工学版), 2023, 57(9): 1903-1914.

Wei-long HE,Ping WANG,Ai-hua ZHANG,Ting-ting LIANG,Qiang-jie MA. On-line quantitative detection of welding arc shape based on object detection. Journal of ZheJiang University (Engineering Science), 2023, 57(9): 1903-1914.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2023.09.022    或    https://www.zjujournals.com/eng/CN/Y2023/V57/I9/1903

图 1  焊接电弧形态检测流程
工况 $ {b_\text{g}} $/mm $ U $/V $ {v_\text{f}}{\text{/(mm}} \cdot {{\text{s}}^{ - 1}}) $ $ v{\text{/(mm}} \cdot {{\text{s}}^{ - 1}}) $
1 5.0 22.5 57.1 8.0
2 5.0 23.0 52.6 8.0
表 1  焊接实验的工艺参数
图 2  焊接电弧图像采集系统示意图
图 3  LUT-Arc数据集中典型焊接电弧图像
图片类型 N
训练集 验证集
约束良好 426 109
约束良好(含火焰) 154 38
约束不佳 333 79
约束不佳(含飞溅) 83 23
总计 996 249
表 2  LUT-Arc数据集焊接电弧类型构成
图 4  基线网络YOLOv5s的结构图
图 5  C3模块和LC3模块的网络结构
图 6  Focus模块和Conv-Maxpool模块的网络结构
图 7  Shufflenet V2模块和AD-Shufflenet V2模块的网络结构
聚类算法 锚框尺寸 $ F $/% $ R_\text{P} $/%
k-means+遗传算法 [11,28 28,11 20,22] 0.831 1
k-means [11,29 29,12 19,21] 0.828 1
k-means+ [11,27 28,11 20,22] 0.831 1
表 3  不同聚类算法的聚类效果对比
图 8  EIoU锚框回归过程示意图
图 9  不同变量对标定方法性能影响的测试结果
图 10  不同参照物实际标定图像
测量对象 $ l $/mm $l_{\rm{m} }$/mm E/%
1 2 3 4 5 6 7
螺丝内径 18.00 18.07 18.03 17.89 18.10 18.04 17.92 18.03 0.37
螺丝外径 24.00 24.10 23.93 23.89 24.09 24.04 24.18 23.94 0.39
螺母内径 20.00 19.85 19.91 20.07 20.04 20.11 20.16 19.97 0.46
螺母外径 22.00 22.06 22.07 22.15 21.92 21.96 22.03 22.03 0.30
焊丝直径 1.60 1.61 1.60 1.59 1.58 1.60 1.61 1.60 0.45
均值 — — — — — — — — 0.39
表 4  不同参照物实际测量结果
图 11  YOLO-SEC的网络结构图
网络名称 ${N_{\rm{PC}}}$/106 FLOPs/109 V/MB P/% R/% mAP0.5/% t/ms
YOLOv5s 7.26 14.8 14.4 96.9 96.2 97.8 12.9
YOLO-SEC-A 7.03 14.6 14.0 96.4 96.2 97.4 12.4
YOLO-SEC-B 3.73 2.0 1.0 96.6 97.7 98.0 4.6
YOLO-SEC-C 3.13 1.9 0.8 95.8 97.3 98.1 4.2
YOLO-SEC-D 2.15 1.1 0.6 96.1 95.0 97.7 3.8
YOLO-SEC-E 2.15 1.1 0.7 96.1 96.2 98.1 4.2
YOLO-SEC-F 1.60 0.6 0.4 97.7 98.8 98.9 3.9
YOLO-SEC 1.60 0.6 0.4 97.7 98.5 99.2 3.1
表 5  YOLO-SEC消融实验的检测指标对比
图 12  不同电弧检测网络的准确率-召回率曲线
图 13  YOLO-SEC与4种轻量化网络检测指标对比
网络名称 ${N_{{\rm{PC}}} }$/106 FLOPs/109 V/MB P/% R/% mAP0.5/% t/ms
SDD 23.75 59.5 90.7 94.5 95.20 95.85 25.0
YOLOv3 61.94 156.3 18.8 94.8 98.50 98.90 125.0
YOLOv4 64.36 60.50 277.7 85.6 91.57 91.70 27.8
YOLOv3-tiny 8.67 13.3 17.4 97.4 98.10 97.60 14.7
YOLOv4-tiny 6.95 6.1 23.0 95.9 98.33 97.70 14.5
YOLOv5 lite 0.80 8.7 2.1 93.4 95.50 95.90 9.8
YOLOv5 Nano 1.86 4.5 3.8 97.3 96.10 98.40 6.1
YOLO-SEC 1.60 0.6 0.4 97.7 98.50 99.20 3.1
表 6  不同目标检测网络的检测指标对比
图 14  YOLO-SEC与4种轻量化网络的实际检测效果对比
图 15  上位机终端显示界面展示
图 16  4种焊接电弧的实际检测效果展示
1 朱志明, 程世佳, 符平坡, 等 焊接电弧行为特征和稳定性评价研究与进展[J]. 焊接, 2018, (11): 1- 6
ZHU Zhi-ming, CHENG Shi-jia, FU Ping-po, et al Research and progress on behavior characteristics and stability evaluation of welding arc[J]. Welding and Joining, 2018, (11): 1- 6
2 MIAO J, LI Y, DONG Z, et al Research progress of low spatter CO2 welding technology [J]. The International Journal of Advanced Manufacturing Technology, 2022, 123: 3715- 3736
doi: 10.1007/s00170-022-10504-9
3 陈欢, 朱亮, 姚仁, 等 超窄间隙焊接中送进式焊剂带约束电弧的加热特性[J]. 电焊机, 2021, 51 (9): 41- 45
CHEN Huan, ZHU Liang, YAO Ren, et al Heating characteristics of continuous feeding flux strips constrained arc in ultra-narrow gap welding[J]. Electric Welding Machine, 2021, 51 (9): 41- 45
4 朱亮, 张旭磊, 郑韶先, 等 焊剂带约束电弧超窄间隙焊接的实现[J]. 兰州理工大学学报, 2007, 33 (3): 27- 30
ZHU Liang, ZHANG Xu-lei, ZHENG Shao-xian, et al Realization of ultra-narrow gap welding with flux strips constraining arc[J]. Journal of Lanzhou University of Technology, 2007, 33 (3): 27- 30
5 龚练, 朱亮, 张爱华 超窄间隙焊接中焊剂片与电弧相 互作用机制分析[J]. 焊接学报, 2017, 38 (12): 57- 60
GONG Lian, ZHU Liang, ZHANG Ai-hua, et al Interaction mechanism of flux band and arc in ultra-narrow gap welding[J]. Transactions of the China Welding Institution, 2017, 38 (12): 57- 60
6 任乐, 张爱华, 常东东 基于电弧声的超窄间隙焊接熔合状态识别[J]. 电焊机, 2021, 51 (7): 11- 16
REN Le, ZHANG Ai-hua, CHANG Dong-dong, et al Fusion state recognition of ultra-narrow gap welding based on arc sound[J]. Electric Welding Machine, 2021, 51 (7): 11- 16
7 YE D, HUA X, ZHANG J, et al Analysis of arc interference and welding stability in twin wire GMA welding[J]. The International Journal of Advanced Manufacturing Technology, 2015, 81: 627- 633
doi: 10.1007/s00170-015-7225-9
8 ZHU Y, WANG Z, LIU R, et al Study on arc behavior and droplet transfer in twin-electrode TIG-MIG indirect arc welding[J]. The International Journal of Advanced Manufacturing Technology, 2022, 120: 6821- 6831
doi: 10.1007/s00170-022-09131-1
9 LI W, JI Y, WU J, et al A modified welding image feature extraction algorithm for rotating arc narrow gap MAG welding[J]. Industrial Robot, 2015, 42 (3): 222- 227
doi: 10.1108/IR-11-2014-0407
10 王聪, 马国红, 姜芙蓉, 等 基于数字形态学的DE-GMAW电弧图像处理[J]. 热加工工艺, 2012, 41 (19): 145- 147
WANG Cong, MA Guo-hong, JIANG Fu-rong, et al DE-GMAW arc image processing based on mathematical morphology[J]. Hot Working Technology, 2012, 41 (19): 145- 147
11 JAMROZIK W, GÓRKA J Assessing MMA welding process stability using machine vision-based arc features tracking system[J]. Sensors, 2021, 21 (1): 84
12 程世佳, 朱志明, 符平坡 基于电弧图像的脉冲TIG焊电弧形态及特征温度演变规律[J]. 清华大学学报: 自然科学版, 2021, 61 (9): 994- 1001
CHENG Shi-jia, ZHU Zhi-ming, FU Ping-po Arc shape variations and characteristic temperatures of pulsed TIG welding arcs based on observed arc images[J]. Journal of Tsinghua University: Science and Technology, 2021, 61 (9): 994- 1001
13 ZHU C, TANG X, HE Y, et al Study on arc characteristics and their influences on weld bead geometry in narrow gap GMAW of 5083 Al-alloy[J]. The International Journal of Advanced Manufacturing Technology, 2017, 90: 2513- 2525
doi: 10.1007/s00170-016-9563-7
14 REN S, HE K, GIRSHICK R, et al Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39 (6): 1137- 1149
doi: 10.1109/TPAMI.2016.2577031
15 LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot MultiBox detector [C]// Proceedings of European Conference on Computer Vision. Amsterdam: Springer, 2016: 21–37.
16 REDMON J, DIVVAL S, GIRSHICK R, et al. You only look once: unified, real-time object detection [C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 779–788.
17 REDMON J, FARHADI A. YOLO9000: better, faster, stronger [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 6517–6525.
18 REDMON J, FARHADI A. YOLOv3: an incremental improvement [EB/OL]. (2018-01-20)[2022-08-12].　https://arxiv.org/pdf/1804.02767v1.pdf.
19 BOCHKOVSKIY A, WANG C Y, LIAO H Y M. Yolov4: optimal speed and accuracy of object detection [EB/OL]. (2020-04-23) [2022-08-12]　https://arxiv.org/pdf/2004.10934.pdf.
20 郝琨, 王阔, 王贝贝 基于改进Mobilenet-YOLOv3的轻量级水下生物检测算法[J]. 浙江大学学报: 工学版, 2022, 56 (8): 1622- 1632
HAO Kun, WANG Kuo, WANG Bei-bei Lightweight underwater biological detection algorithm based on improved Mobilenet-YOLOv3[J]. Journal of Zhejiang University: Engineering Science, 2022, 56 (8): 1622- 1632
21 董红召, 方浩杰, 张楠 旋转框定位的多尺度再生物品目标检测算法[J]. 浙江大学学报: 工学版, 2022, 56 (1): 16- 25
DONG Hong-zhao, FANG Hao-jie, ZHANG Nan Multi-scale object detection algorithm for recycled objects based on rotating block positioning[J]. Journal of Zhejiang University: Engineering Science, 2022, 56 (1): 16- 25
22 ZHANG Y F, REN W, ZHANG Z, et al Focal and efficient IOU loss for accurate bounding box regression[J]. Neurocomputing, 2022, 506: 146- 157
doi: 10.1016/j.neucom.2022.07.042
23 LIU S, QI L, QIN H, et al. Path aggregation network for instance segmentation [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 8759-8768.
24 WANG C Y, LIAO H Y M, WU Y H, et al. CSPNet: a new backbone that can enhance learning capability of CNN [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Seattle: IEEE, 2020: 390-391.
25 MA N, ZHANG X, ZHENG H T, et al. Shufflenet V2: practical guidelines for efficient CNN architecture design [C]// Proceedings of the European Conference on Computer Vision. [S.l.]: Springer, 2018: 122-138.
26 ZHENG Z, WANG P, LIU W, et al. Distance-IoU loss: faster and better learning for bounding box regression [C]// Proceedings of the AAAI Conference on Artificial Intelligence. 2020, 34(7): 12993-13000.
27 尹英杰, 徐德, 张正涛 等 基于单目视觉的平面测量[J]. 电子测量与仪器学报, 2013, 27 (4): 347- 352
YIN Ying-jie, XU De, ZHANG Zheng-tao, et al Plane measurement based on monocular vision[J]. Journal of Electronic Measurement and Instrument, 2013, 27 (4): 347- 352
[1] 金鑫,庄建军,徐子恒. 轻量化YOLOv5s网络车底危险物识别算法[J]. 浙江大学学报(工学版), 2023, 57(8): 1516-1526.
[2] 张艳,孙晶雪,孙叶美,刘树东,王传启. 基于分割注意力与线性变换的轻量化目标检测[J]. 浙江大学学报(工学版), 2023, 57(6): 1195-1204.
[3] 韩俊,袁小平,王准,陈烨. 基于YOLOv5s的无人机密集小目标检测算法[J]. 浙江大学学报(工学版), 2023, 57(6): 1224-1233.
[4] 曾耀,高法钦. 基于改进YOLOv5的电子元件表面缺陷检测算法[J]. 浙江大学学报(工学版), 2023, 57(3): 455-465.
[5] 郝琨,王阔,王贝贝. 基于改进Mobilenet-YOLOv3的轻量级水下生物检测算法[J]. 浙江大学学报(工学版), 2022, 56(8): 1622-1632.
[6] 张娜,戚旭磊,包晓安,吴彪,涂小妹,金瑜婷. 基于优化预测定位的单阶段目标检测算法[J]. 浙江大学学报(工学版), 2022, 56(4): 783-794.
[7] 褚晶辉,史李栋,井佩光,吕卫. 适用于目标检测的上下文感知知识蒸馏网络[J]. 浙江大学学报(工学版), 2022, 56(3): 503-509.
[8] 于楠晶,范晓飚,邓天民,冒国韬. 基于多头自注意力的复杂背景船舶检测算法[J]. 浙江大学学报(工学版), 2022, 56(12): 2392-2402.
[9] 谢誉,包梓群,张娜,吴彪,涂小妹,包晓安. 基于特征优化与深层次融合的目标检测算法[J]. 浙江大学学报(工学版), 2022, 56(12): 2403-2415.
[10] 李飞,胡坤,张勇,王文善,蒋浩. 基于混合域注意力YOLOv4的输送带纵向撕裂多维度检测[J]. 浙江大学学报(工学版), 2022, 56(11): 2156-2167.
[11] 张云佐,郭威,蔡昭权,李文博. 联合多尺度与注意力机制的遥感图像目标检测[J]. 浙江大学学报(工学版), 2022, 56(11): 2215-2223.
[12] 张融,张为. 基于改进GhostNet-FCOS的火灾检测算法[J]. 浙江大学学报(工学版), 2022, 56(10): 1891-1899.
[13] 周金海,周世镒,常阳,吴耿俊,王依川. 基于超宽带雷达基带信号的多人目标跟踪[J]. 浙江大学学报(工学版), 2021, 55(6): 1208-1214.
[14] 徐利锋,黄海帆,丁维龙,范玉雷. 基于改进DenseNet的水果小目标检测[J]. 浙江大学学报(工学版), 2021, 55(2): 377-385.
[15] 辛文斌,郝惠敏,卜明龙,兰媛,黄家海,熊晓燕. 基于ShuffleNetv2-YOLOv3模型的静态手势实时识别方法[J]. 浙江大学学报(工学版), 2021, 55(10): 1815-1824.