Please wait a minute...
浙江大学学报(工学版)
浙江大学学报(工学版)  2022, Vol. 56 Issue (6): 1071-1078    DOI: 10.3785/j.issn.1008-973X.2022.06.003
智能机器人     
基于自适应随动机构的机器人目标跟随
陈宏鑫(),张北,王春香*(),杨明
上海交通大学 自动化系，上海 200240
Robot target following based on adaptive follower mechanism
Hong-xin CHEN(),Bei ZHANG,Chun-xiang WANG*(),Ming YANG
Department of Automation, Shanghai Jiao Tong University, Shanghai 200240, China
 全文: PDF(3024 KB)   HTML
摘要：

针对机器人使用固定传感器容易丢失跟随目标的问题，提出基于自适应随动机构的行人跟随方法.基于任务需求设计随动式感知机构视野评价指标；在传统规划策略的基础上，提出结合底盘方向的改进策略和基于视野深度加权的自适应角度规划策略，改进随动机构的运动目标跟随效果. 为了提高随动RGB-D传感器的行人位置跟踪效果，使用YOLOv3算法进行目标检测，结合三维坐标解算与位置度量匹配，实现多目标位置的实时跟踪. 基于Gazebo仿真环境与RoboMaster机器人，实现机器人行人跟随功能. 所提规划策略能够取得综合最优的评分指标，并实现机器人对运动行人目标稳定的轨迹跟随. 实验结果证明了所提目标跟随方法的有效性.
关键词： 行人跟随;  移动机器人;  RGB-D传感器;  目标跟踪;  自适应角度规划    
Abstract:

A pedestrian following method based on adaptive follower mechanism was proposed focusing on the problem that robots lose targets easily with fixed sensors. Based on task requirements, the field of view evaluation metrics of the follower perception mechanism were designed. On the basis of traditional planning strategies, an improved planning strategy derived from chassis direction and a depth weighting based adaptive angle planning strategy were proposed to improve the moving target following performance of the follower mechanism. To improve pedestrian position tracking with follower RGB-D sensor, the YOLOv3 algorithm was used for target detection, combined with 3D coordinate solving and position matching to achieve real-time tracking of multiple targets. Gazebo simulation platform and RoboMaster robot were used to implement robot's pedestrian following function. The proposed planning strategy is shown to achieve comprehensive optimal metrics and stable trajectory following to moving pedestrian targets, which proves the effectiveness of the target following method.
Key words: pedestrian following    mobile robot    RGB-D sensor    target tracking    adaptive angle planning
收稿日期: 2022-03-14 出版日期: 2022-06-30
CLC:  TP 242.6  
基金资助: 国家自然科学基金资助项目(61873165，62173228，62103261)
通讯作者: 王春香     E-mail: angelochen@sjtu.edu.cn;wangcx@sjtu.edu.cn
作者简介: 陈宏鑫（1998—），男，硕士生，从事智能机器人、无人驾驶研究. orcid.org/0000-0002-9730-7900. E-mail： angelochen@sjtu.edu.cn
服务　　
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章　　
陈宏鑫
张北
王春香
杨明

引用本文:

陈宏鑫,张北,王春香,杨明. 基于自适应随动机构的机器人目标跟随[J]. 浙江大学学报(工学版), 2022, 56(6): 1071-1078.

Hong-xin CHEN,Bei ZHANG,Chun-xiang WANG,Ming YANG. Robot target following based on adaptive follower mechanism. Journal of ZheJiang University (Engineering Science), 2022, 56(6): 1071-1078.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2022.06.003    或    https://www.zjujournals.com/eng/CN/Y2022/V56/I6/1071

图 1  YOLOv3算法中使用的残差网络结构
图 2  三维坐标解算计算流程图
图 3  跟随实验中使用的RoboMaster机器人平台
图 4  基于Gazebo平台的仿真机器人与地图
图 5  基于ROS的软件架构与功能模块
图 6  YOLOv3算法在仿真与实际场景中的行人检测结果
图 7  仿真平台单目标跟踪轨迹与真实轨迹对比
图 8  使用Azure Kinect数据的多目标匹配与跟踪结果
图 9  使用Azure Kinect数据的多目标跟踪轨迹
规划策略 ${\overline S_{\text{t}}}$ ${\overline S_{\text{p}}}$ ${N_{\text{o}}}$ ${\overline S_{\text{a}}}$ ${\overline T_{\text{f} } }/{\rm{s}}$ ${f_{\text{s}}}$
固定相机 0.667 0.798 1.004 1.516 85.6 0
目标伺服 0.997 0.834 1.282 1.895 120.1 82.14
底盘方向 0.946 0.861 1.291 1.871 117.2 19.47
权重视野 0.993 0.848 1.331 1.908 141.4 33.55
表 1  仿真平台上不同传感器角度规划策略的平均评分对比
图 10  仿真与实物机器人的行人跟随实验场景
图 11  跟随实验中目标与机器人的运动轨迹
1 谢嘉, 桑成松, 王世明, 等 智能跟随移动机器人的研究与应用前景综述[J]. 制造业自动化, 2020, 42 (10): 49- 55
XIE Jia, SANG Cheng-song, WANG Shi-ming, et al Overview of research and application prospect of intelligent following mobile robots[J]. Manufacturing Automation, 2020, 42 (10): 49- 55
doi: 10.3969/j.issn.1009-0134.2020.10.012
2 DAM A, VERMA A, PANGI C T, et al. Person following mobile robot using pedestrian dead-reckoning with inertial data of smartphones [C]// 2020 11th International Conference on Computing, Communication and Networking Technologies. Kharagpur: IEEE, 2020: 1-4.
3 TEE K T M, LAU B T, SISWOYO JO H An improved indoor robot human-following navigation model using depth camera, active IR marker and proximity sensors fusion[J]. Robotics, 2018, 7 (1): 4
doi: 10.3390/robotics7010004
4 AQUINO R J C, BELTRAN C K C, FAJARDO J W A, et al. Image processing based human following cart using 360° camera [C]// 2020 International Conference on Electronics and Sustainable Communication Systems. Coimbator: IEEE, 2020: 375-380.
5 张亚斌. 基于ROS的轮式移动机器人行人目标视觉感知与跟随系统研究[D]. 徐州: 中国矿业大学, 2019: 19-25.
Zhang Ya-bin. Study on visual perception and following system for pedestrian target of wheeled mobile robot based on ROS [D]. Xuzhou: China University of Mining and Technology, 2019: 19-25.
6 CHENG X, JIA Y, SU J, et al. Person-following for telepresence robots using web cameras [C]// 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems. Macau: IEEE, 2019: 2096-2101.
7 TSUN M T K, THENG L B, JO H S. Pathfinding decision-making using proximity sensors, depth camera and active IR marker tracking data fusion for human following companion robot [C]// Proceedings of the 2017 International Conference on Information Technology. Singapore: ACM Press, 2017: 12-16.
8 LU Y, YANG M, WANG C, et al. Pedestrian tracking based on laser and image data fusion [C]// 2019 IEEE International Conference on Real-Time Computing and Robotics. Irkutsk: IEEE, 2019: 231-236.
9 ZHANG W, WANG J, GUO X, et al Two-stream RGB-D human detection algorithm based on RFB network[J]. IEEE Access, 2020, 8: 123175- 123181
doi: 10.1109/ACCESS.2020.3007611
10 JAFARI O H, MITZEL D, LEIBE B. Real-time RGB-D based people detection and tracking for mobile robots and head-worn cameras [C]// 2014 IEEE International Conference on Robotics and Automation. Hong Kong: IEEE, 2014: 5636-5643.
11 YEKKEHFALLAH M, YANG M, CAI Z, et al Accurate 3D localization using RGB-TOF camera and IMU for industrial mobile robots[J]. Robotica, 2021, 39 (10): 1816- 1833
doi: 10.1017/S0263574720001526
12 ZHANG B, YANG M, YUAN W, et al. A novel system for guiding unmanned vehicles based on human gesture recognition [C]// 2020 IEEE International Conference on Real-time Computing and Robotics. Asahikawa: IEEE, 2020: 345-350.
13 REDMON J, FARHADI A. YOLOv3: an incremental improvement [EB/OL]. [2021-05-31]. http://arxiv.org/abs/1804.02767.
14 HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
15 BEWLEY A, GE Z, OTT L, et al. Simple online and realtime tracking [C]// 2016 IEEE International Conference on Image Processing. Phoenix: IEEE, 2016: 3464-3468.
16 WOJKE N, BEWLEY A, PAULUS D. Simple online and realtime tracking with a deep association metric [C]// 2017 IEEE International Conference on Image Processing. Beijing: IEEE, 2017: 3645-3649.
17 ZHANG Y, WANG C, WANG X, et al FairMOT: on the fairness of detection and re-identification in multiple object tracking[J]. International Journal of Computer Vision, 2021, 129 (11): 3069- 3087
doi: 10.1007/s11263-021-01513-4
18 PERILLE D, TRUONG A, XIAO X, et al. Benchmarking metric ground navigation [C]// 2020 IEEE International Symposium on Safety, Security, and Rescue Robotics. Abudhabi: IEEE, 2020: 116-121.
19 WEN J, ZHANG X, BI Q, et al. MRPB 1.0: a unified benchmark for the evaluation of mobile robot local planning approaches [C]// 2021 IEEE International Conference on Robotics and Automation. Xi’an: IEEE, 2021: 8238-8244.
[1] 陈小波,陈玲,梁书荣,胡煜. 重尾非高斯定位噪声下鲁棒协同目标跟踪[J]. 浙江大学学报(工学版), 2022, 56(5): 967-976.
[2] 金立生,华强,郭柏苍,谢宪毅,闫福刚,武波涛. 基于优化DeepSort的前方车辆多目标跟踪[J]. 浙江大学学报(工学版), 2021, 55(6): 1056-1064.
[3] 周金海,周世镒,常阳,吴耿俊,王依川. 基于超宽带雷达基带信号的多人目标跟踪[J]. 浙江大学学报(工学版), 2021, 55(6): 1208-1214.
[4] 宋鹏,杨德东,李畅,郭畅. 整体特征通道识别的自适应孪生网络跟踪算法[J]. 浙江大学学报(工学版), 2021, 55(5): 966-975.
[5] 王康豪,殷海兵,黄晓峰. 基于策略梯度的目标跟踪方法[J]. 浙江大学学报(工学版), 2020, 54(10): 1923-1928.
[6] 荆丹翔,韩军,徐志伟,陈鹰. 基于成像声呐的水下多目标跟踪研究[J]. 浙江大学学报(工学版), 2019, 53(4): 753-760.
[7] 王海军, 葛红娟, 张圣燕. 基于核协同表示的快速目标跟踪算法[J]. 浙江大学学报(工学版), 2017, 51(2): 399-407.
[8] 贾松敏,卢迎彬,王丽佳,李秀智,徐涛. 分层特征移动机器人行人跟踪[J]. 浙江大学学报(工学版), 2016, 50(9): 1677-1683.
[9] 丁夏清,杜卓洋,陆逸卿,刘山. 基于混合势场的移动机器人视觉轨迹规划[J]. 浙江大学学报(工学版), 2016, 50(7): 1298-1306.
[10] 卢维, 项志宇, 于海滨, 刘济林. 基于自适应多特征表观模型的目标压缩跟踪[J]. 浙江大学学报(工学版), 2014, 48(12): 2132-2138.
[11] 王安定, 裘渔洋, 王秀萍, 余燕平, 李式巨. 基于Toeplitz降维子矩阵的空间多目标跟踪算法[J]. J4, 2012, 46(4): 739-743.
[12] 戴渊明, 韦巍, 林亦宁. 基于颜色纹理特征的均值漂移目标跟踪算法[J]. J4, 2012, 46(2): 212-217.
[13] 许雪梅, 李丽娴, 张键洋, 倪兰, 黄征宇, 曹建. 透明液体药剂中可见异物跟踪算法[J]. J4, 2012, 46(10): 1822-1830.
[14] 刘辉涛,汪李明,李建龙. 声纳强脉冲干扰的自适应抵消方法[J]. J4, 2011, 45(3): 515-519.
[15] 钱诚, 张三元. 适用于目标跟踪的加权增量子空间学习算法[J]. J4, 2011, 45(12): 2240-2246.