Please wait a minute...
浙江大学学报(工学版)
浙江大学学报(工学版)  2025, Vol. 59 Issue (8): 1574-1582    DOI: 10.3785/j.issn.1008-973X.2025.08.003
机械工程、能源工程     
基于三维视觉的软体机器人实时定位与控制
张弘1,2(),张学成1,王国强1,顾潘龙2,江楠1
1. 浣江实验室 软体机器人与智能器件研究中心，浙江 诸暨 311800
2. 浙江大学 航空航天学院，浙江 杭州 310058
Real-time positioning and control of soft robot based on three-dimensional vision
Hong ZHANG1,2(),Xuecheng ZHANG1,Guoqiang WANG1,Panlong GU2,Nan JIANG1
1. Center for Soft Machines and Smart Devices, Huanjiang Laboratory, Zhuji 311800, China
2. School of Aeronautics and Astronautics, Zhejiang University, Hangzhou 310058, China
 全文: PDF(2000 KB)   HTML
摘要：

针对软体机器人建模困难、控制稳定性差和精度低等问题，设计模块化气动软体机械臂平台，推导得到从驱动空间、虚拟关节空间到工作空间的位置正解模型. 根据实际结构参数的约束生成工作空间数据集，通过构建KD树的三维空间索引结构，有效地完成了位置逆解求解，极大地提高了求解速度. 采用提出的逆解筛选原则，在多个解中选取最优解，增强了气动软体机械臂的控制稳定性及精度. 为了提升机械臂末端的控制精度，开发结合深度学习与RANSAC算法的机械臂末端三维视觉实时定位算法，实现了机械臂末端位姿的闭环控制，与开环控制相比，实现了约4倍的精度提升.
关键词： 软体机械臂;  KD树;  三维视觉;  深度学习;  运动学    
Abstract:

A platform for modular pneumatic soft robotic arm was designed, and a forward position model from the driving space, virtual joint space to the workspace was derived aiming at the problems of difficult modeling, poor control stability and low accuracy of soft robotic. The workspace data set was generated according to the actual structural parameter constraints. The position inverse solution was effectively completed by constructing the three-dimensional spatial index structure of KD tree. Then the solution speed was greatly improved. The proposed inverse solution selection principle was used to select the optimal solution from multiple solutions, which enhanced the control stability and accuracy of the pneumatic soft robotic arm. A real-time 3D visual positioning algorithm for the end of robotic arm combining deep learning and RANSAC algorithm was developed in order to improve the control accuracy of the end of the robot arm. Then closed-loop control of the end position and orientation of robotic arm was achieved, and about four times more precision improvement was achieved compared with open-loop control.
Key words: soft robotic arm    KD tree    three-dimensional vision    deep learning    kinematics
收稿日期: 2024-06-13 出版日期: 2025-07-28
:  TP 393  
基金资助: 浣江实验室专项资助项目（128102-E52201/031）；浙江省自然科学基金资助项目（LGG21F030003, LGG20E050011）.
作者简介: 张弘（1990—），男，助理研究员，从事机器人及传感器技术的研究. orcid.org/ 0009-0006-1402-3109. E-mail：517180508@qq.com
服务　　
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章　　
张弘
张学成
王国强
顾潘龙
江楠

引用本文:

张弘,张学成,王国强,顾潘龙,江楠. 基于三维视觉的软体机器人实时定位与控制[J]. 浙江大学学报(工学版), 2025, 59(8): 1574-1582.

Hong ZHANG,Xuecheng ZHANG,Guoqiang WANG,Panlong GU,Nan JIANG. Real-time positioning and control of soft robot based on three-dimensional vision. Journal of ZheJiang University (Engineering Science), 2025, 59(8): 1574-1582.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2025.08.003    或    https://www.zjujournals.com/eng/CN/Y2025/V59/I8/1574

图 1  气动软体机械臂运动控制的实验平台
名称型号公司
压电比例阀VAB-B-26-D13FESTO
负压气源装置KVP8 PLUS-KB-SKamer
正压气源装置KZP-PFKamer
压力传感器DP-102APanasonic
调压阀AW2000-02SMC
单片机Arduino MEGA2560Arduino LLC
深度相机RealSense D435iIntel
表 1  机器人实验平台的主要硬件
名称型号版本
操作系统Ubuntu20.04
中央处理器英特尔Xeno-W2155
显卡英伟达QuadroP6000
Pytorch—1.12
Cuda—11.0
Python—3.8
表 2  基于深度学习的算法开发环境配置
图 2  软体机械臂的结构图
图 3  软体驱动器变形的原理图
图 4  控制系统的电气逻辑图
参数数值
软体驱动器个数6
最大负压(?96 kPa)条件下的长度/mm70
初始状态长度/mm103
最大正压(96 kPa)条件下的长度/mm195
永磁铁半径/mm7.5
连接盘半径/mm45
法兰盘尺寸/mm210×140×8
气管内径/mm2
横向约束件内径/mm12.5
表 3  软体机械臂的结构参数
图 5  软体机械臂末端位置的正解解析示意图
序号$ {\theta }_{} $$ {d}_{} $$ {a}_{} $$ {\alpha }_{} $
1$ {\varphi }_{i} $$ 0 $$ 0 $$ -\mathrm{\text{π} }/2 $
2$ {\theta }_{i}/2 $$ 0 $$ 0 $$ \mathrm{\text{π} }/2 $
3$ 0 $$ {(2{L}_{i{\mathrm{e}}}}/{{\theta }_{i})}\mathrm{sin}\left({{\theta }_{i}}/{2}\right) $$ 0 $$ -\mathrm{\text{π} }/2 $
4$ {\theta }_{i}/2 $$ 0 $$ 0 $$ \mathrm{\text{π} }/2 $
5$ -{\varphi }_{i} $$ 0 $$ 0 $$ 0 $
表 4  软体机械臂的D-H参数表
图 6  软体机械臂的位置逆解求解流程图
数据量tb/mstk/ms
50 00041.861.99
100 00084.822.98
150 000124.523.34
200 000167.033.55
250 000209.303.98
表 5  不同数据量下逆解求解算法的性能对比
图 7  KD树求位置逆解的简图
图 8  定位环视觉识别的示意图
图 9  定位环点云的预处理图
组别真实坐标/mm拟合中心点平均坐标/mm方差/mm
2(2.5, ?3, 470.4)(2.43, ?2.41, 470.68)0.65
3(?10, ?5, 440.4)(?10.61, ?5.30, 440.24)0.70
4(1.5, ?4.5, 471.2)(2.53, ?3.93, 471.66)1.26
5(?15.4, ?3.2, 456)(?14.79, ?3.44, 455.27)0.98
6(2.6, ?3.5, 469.8)(2.43, ?3.93, 471.66)1.92
7(?15, ?5, 458.4)(?16.01, ?3.61, 458.05)1.75
8(2.8, ?1.8, 469.7)(2.93, ?2.41, 470.68)1.16
9(4, ?3.5, 439.3)(4.44, ?3.51, 439.24)0.45
10(?9.3, ?5.3, 441)(?10.11, ?5.30, 440.24)1.11
表 6  拟合坐标值与真实坐标值的对比
图 10  软体机械臂的闭环控制流程图
图 11  软体机械臂的弯曲运动示意图
图 12  机械臂末端开环的弯曲运动轨迹
图 13  机械臂末端闭环的弯曲运动轨迹
θ2/(°)ts/sv/(rad·s?1)
0~301.190.44
0~601.420.74
0~901.580.99
0~1201.811.15
表 7  弯曲运动实验过程中的机械臂末端速度
图 14  软体机械臂的偏转运动示意图
图 15  机械臂末端的开环偏转运动轨迹
图 16  机械臂末端的闭环偏转运动轨迹
图 17  偏转运动实验过程中的机械臂末端速度
1 LUAN H, WANG M, ZHANG Q, et al Variable stiffness fibers enabled universal and programmable re-foldability strategy for modular soft robotics[J]. Advanced Science, 2024, 11 (10): 2307350
doi: 10.1002/advs.202307350
2 郝天泽, 肖华平, 刘书海, 等 集成化智能软体机器人研究进展[J]. 浙江大学学报: 工学版, 2021, 55 (2): 229- 243
HAO Tianze, XIAO Huaping, LIU Shuhai, et al Research status of integrated intelligent soft robots[J]. Journal of Zhejiang University: Engineering Science, 2021, 55 (2): 229- 243
3 ARACHCHIGE D, PERERA D, MALLIKARACHCHI S, et al Soft steps: exploring quadrupedal locomotion with modular soft robots[J]. IEEE Access, 2023, 11: 63136- 63148
doi: 10.1109/ACCESS.2023.3289156
4 HUANG J D, ZHOU J S, WANG Z Y, et al Modular origami soft robot with the perception of interaction force and body configuration[J]. Advanced Intelligent Systems, 2022, 4 (9): 2200081
doi: 10.1002/aisy.202200081
5 THURUTHEL T G, ANSARI Y, FALOTICO E, et al Control strategies for soft robotic manipulators: a survey[J]. Soft Robotics, 2018, 5 (2): 149- 163
doi: 10.1089/soro.2017.0007
6 QU J T, XU Y N, LI Z K, et al Recent advances on underwater soft robots[J]. Advanced Intelligent Systems, 2024, 6 (2): 2300299
doi: 10.1002/aisy.202300299
7 WANG S, SUN Z Hydrogel and machine learning for soft robots' sensing and signal processing: a review[J]. Journal of Bionic Engineering, 2023, 20 (3): 845- 857
doi: 10.1007/s42235-022-00320-y
8 YIMING B, HAN Y, HAN Z, et al A mechanically robust and versatile liquid-free ionic conductive elastomer[J]. Advanced Materials, 2021, 33 (11): 2006111
doi: 10.1002/adma.202006111
9 SHAO X, CAI Y, YIN S, et al Mechanics of interfacial delamination in deep-sea soft robots under hydrostatic pressure[J]. Journal of Applied Mechanics, 2023, 90 (2): 021009
doi: 10.1115/1.4056214
10 TANG W, ZHANG Y, XU H, et al Self-protection soft fluidic robots with rapid large-area self-healing capabilities[J]. Nature Communications, 2023, 14 (1): 6430
doi: 10.1038/s41467-023-42214-5
11 LI G, CHEN X, ZHOU F, et al Self-powered soft robot in the Mariana Trench[J]. Nature, 2021, 591 (7848): 66- 71
doi: 10.1038/s41586-020-03153-z
12 XU F, WANG H Soft Robotics: morphology and morphology-inspired motion strategy[J]. IEEE-CAA Journal of Automatica Sinica, 2021, 8 (9): 1500- 1522
doi: 10.1109/JAS.2021.1004105
13 VINOCOUR P E T, KOKUBU S, MATSUNAGA F, et al Development of a dual function joint modular soft actuator and its evaluation using a novel dummy finger joint-soft actuator complex model[J]. IEEE Robotics and Automation Letters, 2024, 9 (5): 4210- 4217
doi: 10.1109/LRA.2024.3376975
14 SU M, XIE R, QIU Y, et al Design, mobility analysis and gait planning of a leech-like soft crawling robot with stretching and bending deformation[J]. Journal of Bionic Engineering, 2023, 20 (1): 69- 80
doi: 10.1007/s42235-022-00256-3
15 DAS R, BABU S P M, VISENTIN F, et al An earthworm-like modular soft robot for locomotion in multi-terrain environments[J]. Scientific Reports, 2023, 13 (1): 1571
doi: 10.1038/s41598-023-28873-w
16 ZHANG K, FAN Y, SHEN S Tunable folding assembly strategy for soft pneumatic actuators[J]. Soft Robotics, 2023, 10 (6): 1099- 1114
doi: 10.1089/soro.2022.0166
17 LI G, WONG T W, SHIH B, et al Bioinspired soft robots for deep-sea exploration[J]. Nature Communications, 2023, 14 (1): 7097
doi: 10.1038/s41467-023-42882-3
18 GONG Z, FANG X, CHEN X, et al A soft manipulator for efficient delicate grasping in shallow water: modeling, control, and real-world experiments[J]. International Journal of Robotics Research, 2021, 40 (1): 449- 469
doi: 10.1177/0278364920917203
19 WAN Z, SUN Y, QIN Y, et al Design, analysis, and real-time simulation of a 3D soft robotic snake[J]. Soft Robotics, 2023, 10 (2): 258- 268
doi: 10.1089/soro.2021.0144
20 FOKOU M R T, XIA Q, JIN H, et al A soft robotic fish actuated by artificial muscle modules (SoRoFAAM-1)[J]. Journal of Bionic Engineering, 2023, 20 (5): 2030- 2043
doi: 10.1007/s42235-023-00390-6
21 HABICH T L, HAACK J, BELHADJ M, et al SPONGE: open-source designs of modular articulated soft robots[J]. IEEE Robotics and Automation Letters, 2024, 9 (6): 5346- 5353
doi: 10.1109/LRA.2024.3388855
22 SHAN Y, ZHAO Y, YU H, et al Design and grasping force modeling for a soft robotic gripper with multi-stem twining[J]. Journal of Bionic Engineering, 2023, 20 (5): 2123- 2134
doi: 10.1007/s42235-023-00371-9
23 ZHANG J, LI Y, KAN Z, et al A preprogrammable continuum robot inspired by elephant trunk for dexterous manipulation[J]. Soft Robotics, 2023, 10 (3): 636- 646
doi: 10.1089/soro.2022.0048
24 CHEN G, LIN T, LODEWIJKS G, et al Design of an active flexible spine for wall climbing robot using pneumatic soft actuators[J]. Journal of Bionic Engineering, 2023, 20 (2): 530- 542
doi: 10.1007/s42235-022-00273-2
25 张志远, 王松涛, 王学谦, 等 螺线型气动软体致动器设计与建模[J]. 机器人, 2020, 42 (1): 10- 20
ZHANG Zhiyuan, WANG Songtao, WANG Xueqian, et al Design and modeling of a spiral pneumatic soft actuator[J]. Robot, 2020, 42 (1): 10- 20
26 夏泽洋, 陈君, 甘阳洲, 等 一种用于刚软混杂机器人仿真的耦合模型[J]. 机器人, 2021, 43 (1): 29- 35
XIA Zeyang, CHEN Jun, GAN Yangzhou, et al A coupled model for the simulation of rigid-soft hybrid robot[J]. Robot, 2021, 43 (1): 29- 35
27 王昱欣, 王贺升, 陈卫东 软体机器人手眼视觉/形状混合控制[J]. 机器人, 2018, 40 (5): 619- 625
WANG Yuxin, WANG Hesheng, CHEN Weidong The hybrid eye-in-hand visual/shape control for soft robots[J]. Robot, 2018, 40 (5): 619- 625
28 MULLER D, SAWODNY O Modeling the soft bellows of the bionic soft arm[J]. Ifac Papersonline, 2022, 55 (20): 229- 234
doi: 10.1016/j.ifacol.2022.09.100
29 徐璠, 王贺升 软体机械臂水下自适应鲁棒视觉伺服[J]. 自动化学报, 2023, 49 (4): 744- 753
XU Fan, WANG Hesheng Underwater adaptive robust visual servo for soft manipulator[J]. Acta Automatica Sinica, 2023, 49 (4): 744- 753
30 CHENG D, XU R, ZHANG B, et al Fast density estimation for density-based clustering methods[J]. Neurocomputing, 2023, 532 (1): 170- 182
31 徐志博, 刘永生, 户盼茹 结合离群因子和K-means++聚类改进的点云去噪算法[J]. 信息技术与信息化, 2023, (3): 21- 24
XU Zhibo, LIU Yongsheng, HU Panru Improved denoising algorithm for point cloud based on outlier factor and K-means++ clustering[J]. Information Technology and Informatization, 2023, (3): 21- 24
32 唐佳林, 林寿南, 周壮, 等 基于特征变换结合KD树改进ICP的快速点云配准方法[J]. 计算机科学, 2023, 50 (Suppl.2): 248- 252
TANG Jialin, LIN Shounan, ZHOU Zhuang, et al Fast point cloud registration method for improving ICP based on feature transformation combined with KD tree[J]. Computer Science, 2023, 50 (Suppl.2): 248- 252
[1] 王圣举,张赞. 基于加速扩散模型的缺失值插补算法[J]. 浙江大学学报(工学版), 2025, 59(7): 1471-1480.
[2] 章东平,王大为,何数技,汤斯亮,刘志勇,刘中秋. 基于跨维度特征融合的航空发动机寿命预测[J]. 浙江大学学报(工学版), 2025, 59(7): 1504-1513.
[3] 蔡永青,韩成,权巍,陈兀迪. 基于注意力机制的视觉诱导晕动症评估模型[J]. 浙江大学学报(工学版), 2025, 59(6): 1110-1118.
[4] 王立红,刘新倩,李静,冯志全. 基于联邦学习和时空特征融合的网络入侵检测方法[J]. 浙江大学学报(工学版), 2025, 59(6): 1201-1210.
[5] 徐慧智,王秀青. 基于车辆图像特征的前车距离与速度感知[J]. 浙江大学学报(工学版), 2025, 59(6): 1219-1232.
[6] 陈赞,李冉,冯远静,李永强. 基于时间维超分辨率的视频快照压缩成像重构[J]. 浙江大学学报(工学版), 2025, 59(5): 956-963.
[7] 马莉,王永顺,胡瑶,范磊. 预训练长短时空交错Transformer在交通流预测中的应用[J]. 浙江大学学报(工学版), 2025, 59(4): 669-678.
[8] 陈巧红,郭孟浩,方贤,孙麒. 基于跨模态级联扩散模型的图像描述方法[J]. 浙江大学学报(工学版), 2025, 59(4): 787-794.
[9] 顾正宇,赖菲菲,耿辰,王希明,戴亚康. 基于知识引导的缺血性脑卒中梗死区分割方法[J]. 浙江大学学报(工学版), 2025, 59(4): 814-820.
[10] 姚明辉,王悦燕,吴启亮,牛燕,王聪. 基于小样本人体运动行为识别的孪生网络算法[J]. 浙江大学学报(工学版), 2025, 59(3): 504-511.
[11] 梁礼明,龙鹏威,金家新,李仁杰,曾璐. 基于改进YOLOv8s的钢材表面缺陷检测算法[J]. 浙江大学学报(工学版), 2025, 59(3): 512-522.
[12] 杨凯博,钟铭恩,谭佳威,邓智颖,周梦丽,肖子佶. 基于半监督学习的多场景火灾小规模稀薄烟雾检测[J]. 浙江大学学报(工学版), 2025, 59(3): 546-556.
[13] 陈智超,杨杰,李凡,冯志成. 基于深度学习的列车运行环境感知关键算法研究综述[J]. 浙江大学学报(工学版), 2025, 59(1): 1-17.
[14] 刘登峰,陈世海,郭文静,柴志雷. 基于轻量残差网络的高效半色调算法[J]. 浙江大学学报(工学版), 2025, 59(1): 62-69.
[15] 赵顗,安醇,李铭浩,马健霄,怀硕. 城市快速路互通交织区车辆的换道持续距离选择[J]. 浙江大学学报(工学版), 2025, 59(1): 205-212.