Please wait a minute...
浙江大学学报(工学版)  2024, Vol. 58 Issue (9): 1956-1969    DOI: 10.3785/j.issn.1008-973X.2024.09.020
机械工程     
面向移动作业的腿足机器人数字孪生系统
林俊杰1(),朱雅光1,2,*(),刘春潮1,刘昊洋1
1. 长安大学 道路施工技术与装备教育部重点实验室,陕西 西安 710064
2. 哈尔滨工业大学 机器人技术与系统国家重点实验室,黑龙江 哈尔滨 150001
Digital twin system of legged robot for mobile operation
Junjie LIN1(),Yaguang ZHU1,2,*(),Chunchao LIU1,Haoyang LIU1
1. The Key Laboratory of Road Construction Technology and Equipment of MOE, Chang’an University, Xi’an 710064, China
2. State Key Laboratory of Robotics and Systems, Harbin Institute of Technology, Harbin 150001, China
 全文: PDF(2451 KB)   HTML
摘要:

针对移动作业中的腿足机器人,结合数字孪生技术,设计整机系统的体系结构、模块结构、硬件框架和软件框架,通过集成多个传感器输入和数据源,可在移动场景下获得可靠准确的环境状态和机器人状态. 基于点线特征匹配理论,优化腿足机器人的自主定位精度和鲁棒性,结合环境建模数据实现高效里程计和实时移动建图. 提出建立数字孪生模型的通用方法,并通过误差补偿保证机器人数字孪生体的运动状态与实际机器人的运动状态高度一致. 在数据集和真实机器人上进行实验,结果表明所提出的数字孪生系统不仅能够在不同的腿足机器人平台上稳定高效运行,而且能够保证实时状态反馈和里程计定位精度. 与ORB-SLAM3相比,内存开销降低约68.7%,CPU使用率降低约17.8%. 硬件实验表明,通信延迟与网络延迟基本一致,约为30 ms,有助于提高任务执行效率.

关键词: 数字孪生控制系统腿足机器人自主定位点线特征提取    
Abstract:

A digital twin system for the mobile operation of legged robots was proposed, encompassing the design of architecture, module structure, hardware framework, and software framework. The system enabled reliable and accurate acquisition of environmental states and robot states in mobile scenarios by integrating multiple sensor inputs and data sources. The point and line feature matching theory was used to optimize the autonomous positioning accuracy and the robustness of the legged robot, and the odometer functionality and the real-time mobile mapping were effectively achieved through integration with the environmental modeling data. A general modeling method was introduced to establish a digital twin model that ensured high consistency between the simulated robot motion state and the real robot motion state through error compensation techniques. Experimental results on both datasets and real robots demonstrated that the proposed digital twin system not only operated stably and efficiently across various legged robot platforms but also ensured the real-time state feedback and the odometer positioning accuracy. Compared with ORB-SLAM3, the memory overhead was reduced by about 68.7%, and the CPU usage was reduced by about 17.8%. The hardware experiments showed that the communication delay was basically consistent with the network delay of about 30 ms, which helped to improve the efficiency of task execution.

Key words: digital twin    control system    legged robot    autonomous positioning    point and line feature extraction
收稿日期: 2024-01-01 出版日期: 2024-08-30
CLC:  TP 24  
基金资助: 国家自然科学基金资助项目(62373064);机器人技术与系统国家重点实验室开放基金资助项目(SKLRS-2023-KF-05);中央高校基本科研业务费专项资金资助项目(300102259308, 300102259401).
通讯作者: 朱雅光     E-mail: linjunjie@chd.edu.cn;zhuyaguang@chd.edu.cn
作者简介: 林俊杰(1998—),男,硕士生,从事机器人数字孪生和自主导航研究. orcid.org/0009-0001-3878-6710. E-mail:linjunjie@chd.edu.cn
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章  
林俊杰
朱雅光
刘春潮
刘昊洋

引用本文:

林俊杰,朱雅光,刘春潮,刘昊洋. 面向移动作业的腿足机器人数字孪生系统[J]. 浙江大学学报(工学版), 2024, 58(9): 1956-1969.

Junjie LIN,Yaguang ZHU,Chunchao LIU,Haoyang LIU. Digital twin system of legged robot for mobile operation. Journal of ZheJiang University (Engineering Science), 2024, 58(9): 1956-1969.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2024.09.020        https://www.zjujournals.com/eng/CN/Y2024/V58/I9/1956

图 1  腿足机器人数字孪生系统体系结构
图 2  腿足机器人数字孪生系统模块结构
图 3  腿足机器人数字孪生系统硬件框架
图 4  腿足机器人数字孪生系统软件框架
图 5  机器人惯性坐标与质心坐标系
图 6  腿足机器人整机控制框架
图 7  六足机器人中枢神经网络
图 8  物理和孪生六足机器人不同步态的运动
图 9  四足机器人的物理运动、仿真和数字孪生模型
图 10  空载情况下的仿真、物理关节拟合曲线
图 11  触地条件下增加动力学加不确定性补偿后物理实体与数字孪生体之间的误差
图 12  3种算法的平均CPU占用率和内存占用率
方法算法APERMSE
1VINS-Fusion0.4194680.446491
2ORB-SLAM30.0498370.049837
3本研究算法0.2613310.281762
表 1  3种算法的绝对姿态误差和均方根误差
图 13  六足机器人的实验平台
图 14  机器人里程表、环境模型和实时画面在UI界面的显示
通信方式发送节点接收节点时差/ms
USBIMUSLAM18
LAN激光雷达SLAM16
USB深度相机SLAM23
WIFISLAMUI90
LANSLAM运动控制16
WIFIUI运动控制30
表 2  不同通信方式下各节点之间的通信延迟
1 LU Y Industry 4.0: a survey on technologies, applications and open research issues[J]. Journal of Industrial Information Integration, 2017, 6: 1- 10
doi: 10.1016/j.jii.2017.04.005
2 ZEID I, STEIGER-ESCOBAR S, BOGRAD M, et al. Industry partnership to help transform liberal arts graduates to advanced manufacturing careers [C]// ASME International Mechanical Engineering Congress and Exposition . Houston: American Society of Mechanical Engineers, 2015.
3 GRANGEL-GONZÁLEZ I, HALILAJ L, COSKUN G, et al. Towards a semantic administrative shell for industry 4.0 components [C]// 2016 IEEE 10th International Conference on Semantic Computing . Laguna Hills: IEEE, 2016: 230−237.
4 LI L China's manufacturing locus in 2025: with a comparison of "Made-in-China 2025" and "Industry 4.0"[J]. Technological Forecasting and Social Change, 2018, 135: 66- 74
doi: 10.1016/j.techfore.2017.05.028
5 MALIK A A, BREM A Digital twins for collaborative robots: a case study in human-robot interaction[J]. Robotics and Computer-Integrated Manufacturing, 2021, 68: 102092
doi: 10.1016/j.rcim.2020.102092
6 TUEGEL E J, INGRAFFEA A R, EASON T G, et al Reengineering aircraft structural life prediction using a digital twin[J]. International Journal of Aerospace Engineering, 2011, (1): 154798
7 CERRONE A, HOCHHALTER J, HEBER G, et al On the effects of modeling as-manufactured geometry: toward digital twin[J]. International Journal of Aerospace Engineering, 2014, (1): 439278
8 HUANG Z Q, SHEN Y, LI J Y, et al A survey on AI-driven digital twins in industry 4.0: smart manufacturing and advanced robotics[J]. Sensors, 2021, 21 (19): 6340
doi: 10.3390/s21196340
9 杨艳芳, 贺焕, 舒亮, 等 断路器柔性装配数字孪生机器人及其运动控制[J]. 计算机集成制造系统, 2020, 26 (11): 2915- 2926
YANG Yanfang, HE Huan, SHU Liang, et al Digital twin robot and its motion control for flexible assembly of circuit breaker[J]. Computer Integrated Manufacturing Systems, 2020, 26 (11): 2915- 2926
10 LUMER-KLABBERS G, HAUSTED J O, KVISTGAARD J L, et al. Towards a digital twin framework for autonomous robots [C]// 2021 IEEE 45th Annual Computers, Software, and Applications Conference . Madrid: IEEE, 2021: 1254−1259.
11 LIU H, ZHAO W, LI S, et al Construction method of virtual-real drive systems for robots in digital twin workshops[J]. China Mechanical Engineering, 2022, 33 (21): 2623
12 HOU Z, HE W Modeling and control of digital twin-based aircraft assembly state inspection robot[J]. Computer Integrated Manufacturing Systems, 2021, 27 (4): 981- 989
13 YAMADA T, ABE H, KAWABATA K. Development of testing method considering tasks with remotely controlled robots in Fukushima Daiichi nuclear power station [C]// 2 021 IEEE International Conference on Intelligence and Safety for Robotics . Tokoname: IEEE, 2021: 131−134.
14 GARG G, KUTS V, ANBARJAFARI G Digital twin for fanuc robots: industrial robot programming and simulation using virtual reality[J]. Sustainability, 2021, 13 (18): 10336
doi: 10.3390/su131810336
15 SWEE S K, AL-QUDAH A. Wireless control system for six-legged autonomous insect robot [C]// MATEC Web of Conferences . Cape Town: EDP Sciences, 2016.
16 LIANG S N, TAN K O, CLEMENT T H L, et al. Open source hardware and software platform for robotics and artificial intelligence applications [C]// IOP Conference Series: Materials Science and Engineering . Kuala Lumpur: IOP Publishing, 2016.
17 LIN R, GUO W, LI M, et al. Novel design of a legged mobile lander for extraterrestrial planet exploration [J]. International Journal of Advanced Robotic Systems , 2017, 14(6): 1729881417746120.
18 WANG Y, MA H mvil-fusion: monocular visual-inertial-lidar simultaneous localization and mapping in challenging environments[J]. IEEE Robotics and Automation Letters, 2022, 8 (2): 504- 511
19 LIU Y, LI Z, XIAO L, et al FDO-Calibr: visual-aided IMU calibration based on frequency-domain optimization[J]. Measurement Science and Technology, 2023, 34 (4): 045108
doi: 10.1088/1361-6501/acadfb
20 GOMEZ-OJEDA R, MORENO F A, ZUNIGA-NOËL D, et al PL-SLAM: a stereo SLAM system through the combination of points and line segments[J]. IEEE Transactions on Robotics, 2019, 35 (3): 734- 746
doi: 10.1109/TRO.2019.2899783
21 LU Y, LIU C, KEVIN I, et al Digital twin-driven smart manufacturing: connotation, reference model, applications and research issues[J]. Robotics and Computer-Integrated Manufacturing, 2020, 61: 101837
doi: 10.1016/j.rcim.2019.101837
22 REZAEE H, ABDOLLAHI F A decentralized cooperative control scheme with obstacle avoidance for a team of mobile robots[J]. IEEE Transactions on Industrial Electronics, 2013, 61 (1): 347- 354
23 ZHU Y, ZHANG L, MANOONPONG P Generic mechanism for waveform regulation and synchronization of oscillators: an application for robot behavior diversity generation[J]. IEEE Transactions on Cybernetics, 2020, 52 (6): 4495- 4507
24 ZHU Y, ZHOU S, GAO D, et al Synchronization of non-linear oscillators for neurobiologically inspired control on a bionic parallel waist of legged robot[J]. Frontiers in Neurorobotics, 2019, 13: 59
25 DI Carlo J, WENSING P M, KATZ B, et al. Dynamic locomotion in the mit cheetah 3 through convex model-predictive control [C]// IEEE/RSJ International Conference on Intelligent Robots and Systems . Madrid: IEEE, 2018: 1−9.
26 VILLARREAL O, BARASUOL V, WENSING P M, et al. MPC-based controller with terrain insight for dynamic legged locomotion [C]// IEEE International Conference on Robotics and Automation . Paris: IEEE, 2020: 2436−2442.
27 RIGHETTI L, IJSPEERT A J. Pattern generators with sensory feedback for the control of quadruped locomotion [C]// IEEE International Conference on Robotics and Automation . Pasadena: IEEE, 2008: 819−824.
28 QIN H, ZHU Y, ZHANG Y, et al. Terrain estimation with least squares and virtual model control for quadruped robots [C]// Journal of Physics: Conference Series . Wuhan: IOP Publishing, 2022.
29 QIN T, LI P, SHEN S VINS-Mono: a robust and versatile monocular visual-inertial state estimator[J]. IEEE Transactions on Robotics, 2018, 34 (4): 1004- 1020
doi: 10.1109/TRO.2018.2853729
30 AKINLAR C, TOPAL C. Edlines: real-time line segment detection by edge drawing (ed) [C]// 18th IEEE International Conference on Image Processing . Brussels: IEEE, 2011: 2837−2840.
31 ZHANG L, KOCH R An efficient and robust line segment matching approach based on LBD descriptor and pairwise geometric consistency[J]. Journal of Visual Communication and Image Representation, 2013, 24 (7): 794- 805
doi: 10.1016/j.jvcir.2013.05.006
32 QIN T, SHEN S. Online temporal calibration for monocular visual-inertial systems [C]// IEEE/RSJ International Conference on Intelligent Robots and Systems . Madrid: IEEE, 2018: 3662−3669.
33 CAMPOS C, ELVIRA R, RODRÍGUEZ J J G, et al Orb-slam3: an accurate open-source library for visual, visual-inertial, and multimap slam[J]. IEEE Transactions on Robotics, 2021, 37 (6): 1874- 1890
doi: 10.1109/TRO.2021.3075644
[1] 丁三波,张康,杨飞生,张家安. 基于动态量化的电力系统事件触发负荷频率控制[J]. 浙江大学学报(工学版), 2023, 57(7): 1460-1469.
[2] 赵永胜,赵志勇,李迎,张涛. 基于数字孪生的机身对接精度优化控制方法[J]. 浙江大学学报(工学版), 2023, 57(5): 883-891.
[3] 赵永胜,李瑞祥,牛娜娜,赵志勇. 数字孪生驱动的机身形状控制方法[J]. 浙江大学学报(工学版), 2022, 56(7): 1457-1463.
[4] 冷柏寒,夏唐斌,孙贺,王皓,奚立峰. 面向可重构制造的数字孪生映射建模与监控仿真[J]. 浙江大学学报(工学版), 2022, 56(5): 843-855.
[5] 李琳利,顾复,李浩,顾新建,罗国富,武志强,刚轶金. 仿生视角的数字孪生系统信息安全框架及技术[J]. 浙江大学学报(工学版), 2022, 56(3): 419-435.
[6] 程浙武,童水光,童哲铭,张钦国. 工业锅炉数字化设计与数字孪生综述[J]. 浙江大学学报(工学版), 2021, 55(8): 1518-1528.
[7] 郑守国,张勇德,谢文添,樊虎,王青. 基于数字孪生的飞机总装生产线建模[J]. 浙江大学学报(工学版), 2021, 55(5): 843-854.
[8] 孟祥飞,王仁广,徐元利. 双行星排汽车纯电驱动模式的转矩分配策略[J]. 浙江大学学报(工学版), 2020, 54(11): 2214-2223.
[9] 师建鹏, 权龙, 张晓刚, 熊晓燕. 进出口独立复合控制挖掘机的动臂速度位置特性[J]. 浙江大学学报(工学版), 2017, 51(9): 1797-1807.
[10] 丁孺琦, 徐兵, 张军辉. 负载口独立控制系统压力速度复合控制的耦合特性[J]. 浙江大学学报(工学版), 2017, 51(6): 1126-1134.
[11] 孙伟, 杜家楠, 王林涛, 马宏辉. 盾构管片拼装机电液系统高速-低冲击控制方法[J]. 浙江大学学报(工学版), 2017, 51(10): 1948-1958.
[12] 梁耀,冯冬芹. 基于攻击增益的工业控制系统物理层安全风险评估[J]. 浙江大学学报(工学版), 2016, 50(3): 589-.
[13] 刘刚, 靳立强, 王熠. 乘用车稳定性自抗扰控制策略[J]. 浙江大学学报(工学版), 2016, 50(12): 2289-2296.
[14] 高梦州, 冯冬芹, 凌从礼, 褚健. 基于攻击图的工业控制系统脆弱性分析[J]. 浙江大学学报(工学版), 2014, 48(12): 2123-2131.
[15] 陈旭东,程放,冯攀. 高阶控制系统动态性能分析[J]. J4, 2014, 48(1): 141-148.