Please wait a minute...
浙江大学学报(工学版)
Journal of ZheJiang University (Engineering Science)  2021, Vol. 55 Issue (1): 145-152    DOI: 10.3785/j.issn.1008-973X.2021.01.017
    
Online correction algorithm for posture by robot assembly based on machine vision
Da-zhao DONG1,2(),Guan-hua XU1,4,*(),Ji-liang GAO3,Yue-tong XU1,Jian-zhong FU1
1. Key Laboratory of 3D Printing Process and Equipment of Zhejiang Province, State Key Laboratory of Fluid Power and Mechatronic Systems, Zhejiang University, Hangzhou 310027, China
2. Polytechnic Institute, Zhejiang University, Hangzhou 310027, China
3. Suzhou Xinzhi Mechatronics Technology Limited Company, Suzhou 215101, China
4. Suzhou Zijingang Intelligent Manufacturing Equipment Limited Company, Kunshan 215300, China
Download: HTML     PDF(1857KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

A robot assembly posture online correction algorithm based on machine vision was proposed in order to solve the problem that the relative position between the workpiece and fixture was uncertain in the process of clamping special-shaped parts with flexible fixture. A workpiece pose vector was established by image preprocessing and surface feature extracting. The correction of posture was decomposed into the original posture error, the posture error introduced by rotation and the residual posture error by system modeling, error analysis and function fitting. Linear combination of these three corrections was fed back to the robot as an error compensation for the posture error of the workpiece in order to guide the robot to complete the assembly task. Experiment was conducted by taking the assembly process for the rotating scroll of automotive aircon scroll compressor, and a robot hand-eye assembly system was established in order to verify the effectiveness of the method. The experimental results showed that the angular deviation and the displacement deviation in the x and y directions between the corrected posture and target posture were less than 0.6 degrees and 0.6 mm, respectively. The average assembly time was less than 20 seconds, and the assembly success rate in the experiment was 99.67%.

Key words： industrial robot      automatic assembly      vision-guided      pose detection      error analysis      posture correction     
Received: 17 June 2020      Published: 05 January 2021
CLC:  TP 242  
Corresponding Authors: Guan-hua XU     E-mail: dazhao_dong@163.com;xgh_zju@163.com
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Da-zhao DONG
Guan-hua XU
Ji-liang GAO
Yue-tong XU
Jian-zhong FU
Cite this article:

Da-zhao DONG,Guan-hua XU,Ji-liang GAO,Yue-tong XU,Jian-zhong FU. Online correction algorithm for posture by robot assembly based on machine vision. Journal of ZheJiang University (Engineering Science), 2021, 55(1): 145-152.

URL:

http://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2021.01.017     OR     http://www.zjujournals.com/eng/Y2021/V55/I1/145


基于机器视觉的机器人装配位姿在线校正算法

为了解决柔性夹具夹取异形零件过程中工件与夹具相对位置具有不确定性的难题，提出基于机器视觉的机器人装配位姿在线校正算法. 通过图像预处理及零件表面特征提取，建立工件位姿向量. 通过系统建模、误差分析及函数拟合，将工件位姿校正量分解为原始位姿差、旋转引入位姿差及残余位姿差三部分，将三部分位姿差进行线性组合作为零件位姿误差补偿量反馈给机器人，以引导机器人完成装配. 为了验证算法的有效性，以涡旋式汽车空调压缩机动盘装配为例，设计机器人手眼装配系统进行实验. 实验结果表明，该系统能够保证校正后的零件位姿与目标位姿角度偏差和x及y方向位置偏差分别小于0.6°和0.6 mm，平均装配时间小于20 s，实验过程中装配成功率达到99.67%.


关键词： 工业机器人,  自动装配,  视觉引导,  位姿检测,  误差分析,  姿态校正 
Fig.1 Schematic diagram of parts and assembly
Fig.2 Model diagram of eye-in-hand assembly system
Fig.3 Image processing flow chart
Fig.4 Surface feature of rotating scroll
Fig.5 Coordinate diagram of hand-eye system
Fig.6 Model of posture error introduced by rotation
$\Delta ^{{\rm{b}}}x$ $\Delta ^{ {\rm{b} } }{{y} }$ ${k_1}$ ${k_2}$
$\Delta ^{{\rm{b}}}x\geqslant 0$ $\Delta ^{ {\rm{b} } }{{y} }\geqslant 0$ 0 1
$\Delta ^{{\rm{b}}}x\geqslant 0$ $\Delta ^{ {\rm{b} } }{{y} } < 0$ 2 ?1
$\Delta ^{{\rm{b}}}x < 0$ $\Delta ^{ {\rm{b} } }{{y} }\geqslant 0$ 1 ?1
$\Delta ^{{\rm{b}}}x < 0$ $\Delta ^{ {\rm{b} } }{{y} } < 0$ 1 1
Tab.1 Value table of ${k}_{1}\;{\text{and}}\;{k}_{2}$
Fig.8 Flow chart of pose correction algorithm
Fig.7 Error model of center of rotation
Fig.9 Example diagram of eye-in-hand assembly system
Fig.10 Measurement results of angular component of residual pose difference
Fig.11 Measurement results of positional component of residual pose difference
Fig.12 Fitting results of $\Delta {x_3}$ changed with correction angle
Fig.13 Fitting results $\Delta {y_3}$ changed with correction angle
项目 ${a_i}$ ${b_i}$ ${\omega _i}$ ${c_i}$
拟合 $\Delta {x}_{3}(i=0)$ 1.161 ?0.3529 0.01923 0.2911
拟合 $\Delta { {{y} } }_{3}(i=1)$ 1.246 0.2909 0.02011 ?1.155
Tab.2 Function parameter list
Fig.14 Standardized residuals diagram of position error fitting
Fig.15 Assembly system position error diagram
[1]   QIAN F Smart and optimal manufacturing: the key for the transformation and development of the process industry[J]. Engineering, 2017, 3 (2): 151
doi: 10.1016/J.ENG.2017.02.016
[2]   FELDMANN K, SLAMA S Highly flexible assembly–scope and justification[J]. CIRP Annals-Manufacturing Technology, 2001, 50 (2): 489- 498
doi: 10.1016/S0007-8506(07)62987-4
[3]   王田苗, 陶永 我国工业机器人技术现状与产业化发展战略[J]. 机械工程学报, 2014, 50 (9): 1- 13
WANG Tian-miao, TAO Yong Research status and industrialization development strategy of Chinese industrial robot[J]. Journal of Mechanical Engineering, 2014, 50 (9): 1- 13
doi: 10.3901/JME.2014.09.001
[4]   BONG G M, CAPSON D Vision-guided fixtureless assembly of automotive components[J]. Robotics and Computer-Integrated Manufacturing, 2003, 19 (1/2): 79- 87
[5]   JIANG J, HUANG Z, BI Z, et al State-of-the-art control strategies for robotic PiH assembly[J]. Robotics and Computer-Integrated Manufacturing, 2020, 65: 101894
doi: 10.1016/j.rcim.2019.101894
[6]   PAGANO S, RUSSO R, SAVINO S A vision guided robotic system for flexible gluing process in the footwear industry[J]. Robotics and Computer-Integrated Manufacturing, 2020, 65: 101965
doi: 10.1016/j.rcim.2020.101965
[7]   GOLNABI H, ASADPOUR A Design and application of industrial machine vision systems[J]. Robotics and Computer Integrated Manufacturing, 2007, 23 (6): 630- 637
doi: 10.1016/j.rcim.2007.02.005
[8]   CHANG W C Robotic assembly of Smartphone back shells with eye-in-hand visual servoing[J]. Robotics and Computer-Integrated Manufacturing, 2017, 50: 102- 113
[9]   WANG W Q, LOU Y, YANG K, et al. Multi-angle automotive fuse box detection and assembly method based on machine vision[J]. Measurement, 2019, 145: 234- 243
doi: 10.1016/j.measurement.2019.05.100
[10]   SEMIM R C, ROSSO R S U, SILVA A G, et al Engine head blocks handling robot guided by vision system[J]. IFAC Proceedings Volumes, 2012, 45 (6): 859- 864
doi: 10.3182/20120523-3-RO-2023.00244
[11]   BORANGIU T, IVANESCU N A, BARAD S Robotized flange assembling with line scan camera control[J]. IFAC Proceedings Volumes, 2003, 36 (23): 119- 124
doi: 10.1016/S1474-6670(17)37672-3
[12]   NERAKAE P, UANGPAIROJ P, CHAMNIPRASART K Using machine vision for flexible automatic assembly system[J]. Procedia Computer Science, 2016, 96: 428- 435
doi: 10.1016/j.procs.2016.08.090
[13]   钟德星, 杨元, 刘瑞玲, 等 基于单目视觉的装配机器人研究及应用[J]. 西安交通大学学报, 2018, 52 (5): 81- 87
ZHONG De-xing, YANG Yuan, LIU Rui-ling, et al Study and application of monocular vision-based assembly robot[J]. Journal of Xi'an Jiaotong University, 2018, 52 (5): 81- 87
[14]   JIANG T, CHENG X S, CUI H H, et al Dual-camera-based method for identification and location of scattered self-plugging rivets for robot grasping[J]. Measurement, 2019, 134: 688- 697
doi: 10.1016/j.measurement.2018.11.017
[15]   HUANG Y, LEE F F An automatic machine vision-guided grasping system for Phalaenopsis tissue culture plantlets[J]. Computers and Electronics in Agriculture, 2010, 70 (1): 42- 51
doi: 10.1016/j.compag.2009.08.011
[16]   GUO D, SUN F C, FANG B, et al Robotic grasping using visual and tactile sensing[J]. Information Sciences, Information Sciences, 2017, 417: 274- 286
[17]   刘毅, 丛明, 刘东, 等 基于改进遗传算法与机器视觉的工业机器人猪腹剖切轨迹规划[J]. 机器人, 2017, 39 (3): 377- 384
LIU Yi, CONG Ming, LIU Dong, et al Trajectory planning for porcine abdomen cutting based on an improved genetic algorithm and machine vision for industrial robot[J]. Robot, 2017, 39 (3): 377- 384
[18]   ZHANG G, YUN T J, OH W B, et al A study on seam tracking in robotic GMA welding process[J]. Materials Today: Proceedings, 2020, 22: 1771- 1777
doi: 10.1016/j.matpr.2020.03.010
[19]   ABDULLAH M W, ROTH H, WEYRICH M, et al An approach for peg-in-hole assembling using intuitive search algorithm based on human behavior and carried by sensors guided industrial robot[J]. IFAC PapersOnLine, 2015, 48 (3): 1476- 1481
doi: 10.1016/j.ifacol.2015.06.295
[20]   张思思, 李凤鸣, 杨旭亭, 等. 基于接触状态感知发育的机器人柔性装配方法[EB/OL]. [2020-10-22]. https://doi.org/10.13195/j.kzyjc.2019.1079.
ZHANG Si-si, LI Feng-ming, YANG Xu-ting, et al. Flexible assembly method based on contact state perception development [EB/OL]. [2020-10-22]. https://doi.org/10.13195/j.kzyjc.2019.1079.
[21]   JASIM I F, PLAPPER P W, VOOS H Position identification in force-guided robotic peg-in-hole assembly tasks[J]. Procedia CIRP, 2014, 23: 217- 222
doi: 10.1016/j.procir.2014.10.077
[22]   NIU L C, SAARINEN M, TUOKKO R, et al Integration of multi-camera vision system for automatic robotic assembly[J]. Procedia Manufacturing, 2019, 37: 380- 384
doi: 10.1016/j.promfg.2019.12.063
[23]   PINTO L, GUPTA A. Supersizing self-supervision: learning to grasp from 50k tries and 700 robot hours [C] // IEEE International Conference on Robotics and Automation. Stockholm: IEEE, 2016: 1544–1551.
[24]   孟少华, 胡瑞钦, 张立建, 等 一种基于机器人的航天器大型部件自主装配方法[J]. 机器人, 2018, 40 (1): 81- 88
MENG Shao-hua, HU Rui-qin, ZHANG Li-jian, et al A method of autonomous assembly of large spacecraft components using robot[J]. Robot, 2018, 40 (1): 81- 88
[25]   马春英, 杜鹃, 郑璟, 等 洪峰流量与流域面积幂函数关系最优拟合方法探讨[J]. 北京师范大学学报: 自然科学版, 2019, 55 (3): 408- 414
MA Chun-ying, DU Juan, ZHENG Jing, et al Optimal fitting for discharge-area power law relationship[J]. Journal of Beijing Normal University: Natural Science, 2019, 55 (3): 408- 414
[1] Ying-jie GUO,Fan GU,Hui-yue DONG,Hai-jin WANG. Prediction and compensation of robot deformation under pressure force of pressure foot[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(8): 1457-1465.
[2] Yi-xiong FENG,Kang-jie LI,Yi-cong GAO,Hao ZHENG. Corner recognition of industrial robot contour curve for visual servoing[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(8): 1449-1456.
[3] Chen-tao MAO,Zhang-wei CHEN,Xiang ZHANG,Hong-fei ZU. Kinematic calibration for robots based on relative accuracy[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(7): 1316-1324.
[4] Chen-yang PU,Zuo-jun LIU,Shuang PANG,Yan ZHANG. Research and application of iterative learning control with knowledge inheritance[J]. Journal of ZheJiang University (Engineering Science), 2019, 53(7): 1340-1348.
[5] HUANG Zi-liang, OUYANG Xiao-ping, ZHAO Tian-fei, ZHANG Jian-bo, ZHOU Liang, YANG Hua-yong. Characteristic of air content detection system for aircraft hydraulics[J]. Journal of ZheJiang University (Engineering Science), 2019, 53(1): 158-165.
[6] CHEN Xing-yu, HUANG Shan-he, He Hao-zhe. Measurement error due to frequency selection in multi-frequency suspended sediment measurement system[J]. Journal of ZheJiang University (Engineering Science), 2018, 52(2): 307-316.
[7] DONG Hui-yue, SUN Qiang, GUO Ying-jie, ZHAO An-an, ZHU Wei-dong. Vibration analysis and suppression in robotic hole chamfering process[J]. Journal of ZheJiang University (Engineering Science), 2018, 52(12): 2243-2252.
[8] WANG Chen-xue, PING Xue-liang, XU Chao. Closed loop calibration of industrial robot for solving constraint plane wandering[J]. Journal of ZheJiang University (Engineering Science), 2018, 52(11): 2110-2119.
[9] HE Xue-jun, WANG Jin, LU Guo-dong, LIU Zhen-yu, CHEN Li, JIN Jing. 3D head portrait sculpture by industrial robot based on triangular mesh slicing and collision detection[J]. Journal of ZheJiang University (Engineering Science), 2017, 51(6): 1104-1110.
[10] HE Xue jun, WANG Jin, LU Guo dong, CHEN Li. Contour-parallel tool path linking method for sections with lake-including islands[J]. Journal of ZheJiang University (Engineering Science), 2016, 50(9): 1654-1661.
[11] HE Xue-jun, WANG Jin, LU Guo-dong, CHEN Li.
Optimization of robot image drawing sequence based on ant colony algorithm
[J]. Journal of ZheJiang University (Engineering Science), 2015, 49(6): 1139-1145.
[12] DONG Hui yue, ZHU Ling sheng, ZHANG Ming, LI Shao bo, LUO Shui jun. Orbital milling method of aircraft skins trimming[J]. Journal of ZheJiang University (Engineering Science), 2015, 49(11): 2033-2039.
[13] LUO Yong-jie, YANG Yong-ying, TIAN Chao, WEI Tao, ZHUO Yong-mo. Error analysis and processing of partial compensatory
aspheric testing system[J]. Journal of ZheJiang University (Engineering Science), 2012, 46(4): 636-642.
[14] LIN Yuan-fang, HUANG Qiang-sheng, RU Qi-tian, SUN Shuo, ZHENG Xiao-dong. Impact of ellipsometric parameters measurement error on
measurement accuracy of thin film parameters[J]. Journal of ZheJiang University (Engineering Science), 2011, 45(8): 1485-1489.
[15] LIU Chu-Hui, TAO Bao-Guo, KE Yang-Lin. Study on offline programming of industrial robot for cutting process[J]. Journal of ZheJiang University (Engineering Science), 2010, 44(3): 426-431.