Please wait a minute...
Journal of ZheJiang University (Engineering Science)  2022, Vol. 56 Issue (6): 1055-1061    DOI: 10.3785/j.issn.1008-973X.2022.06.001
    
Decentralized swarm control based on graph convolutional imitation learning
Ce GUO(),Zhi-wen ZENG*(),Peng-ming ZHU,Zhi-qian ZHOU,Hui-min LU
College of Intelligence Science and Technology, National University of Defense Technology, Changsha 410073, China
Download: HTML     PDF(894KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

A distributed swarm control strategy based on graph convolutional imitation learning was proposed to deal with the cooperative control of robot swarms under restricted communication conditions. The strategy aimed to improve swarm robustness and enhance the success rate of avoiding swarm splitting based on achieving intra-swarm obstacle avoidance and velocity consistency. A quantitative evaluation index of swarm robustness based on entropy evaluation was proposed to establish the connection between the balanced distribution of node and link importance and cluster robustness. The importance-correlated graph convolutional networks were proposed to realize feature extraction and weighted aggregation of non-Euclidean data under restricted communication conditions. A centralized expert strategy was designed to improve swarm robustness, and the graph convolutional imitation learning method was adopted. Furthermore, a distributed swarm cooperative control strategy was obtained by imitating the centralized expert strategy. Simulation experiments demonstrate that the resulting distributed strategy achieves control effects close to those of the centralized expert strategy based on restricted communication conditions.



Key wordsrobot swarm      graph convolutional network      imitation learning      robustness      graph importance entropy     
Received: 27 February 2022      Published: 30 June 2022
CLC:  TP 242.6  
Fund:  国家自然科学基金资助项目(U1913202, U1813205)
Corresponding Authors: Zhi-wen ZENG     E-mail: guoce1997@foxmail.com;z7z7w7@126.com
Cite this article:

Ce GUO,Zhi-wen ZENG,Peng-ming ZHU,Zhi-qian ZHOU,Hui-min LU. Decentralized swarm control based on graph convolutional imitation learning. Journal of ZheJiang University (Engineering Science), 2022, 56(6): 1055-1061.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2022.06.001     OR     https://www.zjujournals.com/eng/Y2022/V56/I6/1055


基于图卷积模仿学习的分布式群集控制

针对受限通信条件下机器人群集协同控制问题,提出基于图卷积模仿学习的分布式群集控制策略. 该策略旨在实现群集内避障、速度一致性的基础上,提高群集鲁棒性,提升避免群集分裂的成功率. 提出基于熵评价的群集鲁棒性量化评价指标,建立节点和链路重要性的均衡分布与群集鲁棒性的联系. 提出重要度相关图卷积网络,用于实现受限通信条件下非欧氏数据的特征提取和加权聚合. 采用图卷积模仿学习方法,根据提升群集鲁棒性的要求设计集中式专家策略,通过对集中式专家策略的模仿,得到分布式群集协同控制策略. 设计仿真实验,证明所得的分布式策略基于受限通信条件实现了接近集中式的专家策略的控制效果.


关键词: 机器人群集,  图卷积网络,  模仿学习,  鲁棒性,  图重要度熵 
Fig.1 Framework for imitation learning based on importance-correlated graph convolutional networks
Fig.2 Structure of importance-correlated graph convolution networks
群集控制策略 $ \stackrel{-}{\sigma } $ $ \stackrel{-}{{E}_{\mathrm{G}}} $ Acc
集中式专家策略 ?52.878 1.891 1.000
IGCNs ?126.183 1.876 0.910
DAGNNs ?225.831 1.669 0.755
分布式专家策略 ?1199.251 1.215 0.080
Tab.1 Quantitative evaluation of swarm control strategies
Fig.3 Initial and stable formations of robot swarm under control of different strategies
Fig.4 Variation of step quantitative evaluation value in selected episode
[1]   GARATTONI L, BIRATTARI M Autonomous task sequencing in a robot swarm[J]. Science Robotics, 2018, 3 (20): eaat0430
doi: 10.1126/scirobotics.aat0430
[2]   VÁSÁRHELYI G, VIRÁGH C, SOMORIAI G, et al Optimized flocking of autonomous drones in confined environments[J]. Science Robotics, 2018, 3 (20): eaat3536
doi: 10.1126/scirobotics.aat3536
[3]   KHALDI B, HARROU F, CHERIF F, et al Self organization in aggregating robot swarms: a DW-KNN topological approach[J]. Biosystems, 2018, 165: 106- 121
doi: 10.1016/j.biosystems.2018.01.005
[4]   YAN L, STOURAITIS T, VIJAYKUMAR S Decentralized ability-aware adaptive control for multi-robot collaborative manipulation[J]. IEEE Robotics and Automation Letters, 2021, 6 (2): 2311- 2318
doi: 10.1109/LRA.2021.3060379
[5]   ZHOU L, TOKEKAR P Multi-robot coordination and planning in uncertain and adversarial environments[J]. Current Robotics Reports, 2021, 2 (2): 147- 157
doi: 10.1007/s43154-021-00046-5
[6]   ZHOU Z, YAO W, MA J, et al. Simatch: a simulation system for highly dynamic confrontations between multi-robot systems [C]// 2018 Chinese Automation Congress. Xi’an: IEEE, 2018: 3934-3939.
[7]   REYNOLDS C W. Flocks, herds and schools: a distributed behavioral model [C]// Proceedings of the 14th annual conference on Computer graphics and interactive techniques. New York: Association for Computing Machinery, 1987: 25-34.
[8]   PROROK A, KUMAR V. Privacy-preserving vehicle assignment for mobility-on-demand systems [C]// 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems. Vancouver: IEEE, 2017: 1869-1876.
[9]   ZHENG H, PANERATI J, BELTRAME G, et al An adversarial approach to private flocking in mobile robot teams[J]. IEEE Robotics and Automation Letters, 2020, 5 (2): 1009- 1016
doi: 10.1109/LRA.2020.2967331
[10]   LI L, BAYUELO A, BOBADILLA L, et al. Coordinated multi-robot planning while preserving individual privacy [C]// 2019 International Conference on Robotics and Automation. Montreal: IEEE, 2019: 2188-2194.
[11]   LI Q, GAMA F, RIBEIRO A, et al. Graph neural networks for decentralized multi-robot path planning [C]// 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems. Las Vegas: IEEE, 2020: 11785-11792.
[12]   TOLSTAYA E, GAMA F, PAULOS J, et al. Learning decentralized controllers for robot swarms with graph neural networks [C]// Conference on robot learning. Osaka: PMLR. 2020: 671-682.
[13]   ZHU P, DAI W, YAO W, et al Multi-robot flocking control based on deep reinforcement learning[J]. IEEE Access, 2020, 8: 150397- 150406
doi: 10.1109/ACCESS.2020.3016951
[14]   WANG Z, GOMBOLAY M Learning scheduling policies for multi-robot coordination with graph attention networks[J]. IEEE Robotics and Automation Letters, 2020, 5 (3): 4509- 4516
doi: 10.1109/LRA.2020.3002198
[15]   HU T K, GAMA F, CHEN T, et al. VGAI: end-to-end learning of vision-based decentralized controllers for robot swarms [C]// ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing. Toronto: IEEE, 2021: 4900-4904.
[16]   GAMA F, TOLSTAYA E, RIBEIRO A. Graph neural networks for decentralized controllers [C]// ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing. Toronto: IEEE, 2021: 5260-5264.
[17]   WANG Z, LIU C, GOMBOLAY M Heterogeneous graph attention networks for scalable multi-robot scheduling with temporospatial constraints[J]. Autonomous Robots, 2022, 46 (1): 249- 268
doi: 10.1007/s10514-021-09997-2
[18]   JIANG Y, HU A, HE M. Evaluation method for the network reliability based on the entropy measures [C]// 2009 International Conference on Networks Security, Wireless Communications and Trusted Computing. Wuhan: IEEE, 2009, 2: 423-426.
[19]   TANNER H G, JADBABAIE A, PAPPAS G J. Stable flocking of mobile agents, part I: fixed topology [C]// 42nd IEEE International Conference on Decision and Control. Maui: IEEE, 2003, 2: 2010-2015.
[1] Xiao-bo CHEN,Ling CHEN,Shu-rong LIANG,Yu HU. Robust cooperative target tracking under heavy-tailed non-Gaussian localization noise[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(5): 967-976.
[2] You-wei WANG,Shuang TONG,Li-zhou FENG,Jian-ming ZHU,Yang LI,Fu CHEN. New inductive microblog rumor detection method based on graph convolutional network[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(5): 956-966.
[3] Ting WANG,Xiao-fei ZHU,Gu TANG. Knowledge-enhanced graph convolutional neural networks for text classification[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(2): 322-328.
[4] Zi-lin TANG,Xiao GAO,Xiao-hui XIAO. Variable stiffiness control for human-robot cooperative transportation based on imitation learning[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(11): 2091-2099.
[5] Qi SHEN,Yan ZHAO,Xiao-wei ZHOU,Xiao-ran YUAN. Image Hashing algorithm based on structure and gradient[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(8): 1525-1533.
[6] Na LV,Chuang LIU,Ke-fan CHEN,Fang-bo CAO. Software defined airborne network election algorithm considering controller failure[J]. Journal of ZheJiang University (Engineering Science), 2019, 53(4): 785-793.
[7] Hao SUI,Gao-feng QIN,Xiang-bo CUI,Xin-jiang LU. Robust fuzzy T-S modeling method based on minimizing mean and variance of modeling error[J]. Journal of ZheJiang University (Engineering Science), 2019, 53(2): 382-387.
[8] Si CHEN,Xiao-dong CAI,Zhen-zhen HOU,Bo LI. Aggregate graph embedding method based on non-uniform neighbor nodes sampling[J]. Journal of ZheJiang University (Engineering Science), 2019, 53(11): 2163-2167.
[9] YANG Chun ning, FANG Jia wei, LI Chun, GE Hui. Hypersonic vehicle blended control methodology based on stability criterion[J]. Journal of ZheJiang University (Engineering Science), 2017, 51(2): 422-428.
[10] LI Tao, WANG Shi-tong. Incremental zero-order TSK fuzzy classifier and its robust version[J]. Journal of ZheJiang University (Engineering Science), 2017, 51(10): 1901-1911.
[11] YANG Hui lin, HUANG Zhi gang, LIU Jiu wen, DU Yuan feng. WIFI fingerprinting localization based on Kernel Fuzzy C means II Clustering[J]. Journal of ZheJiang University (Engineering Science), 2016, 50(6): 1126-1133.
[12] ZHAO Chan yuan, LU Zhi qiang, CUI Wei wei. Proactive scheduling optimization on flow shops with random machine breakdowns[J]. Journal of ZheJiang University (Engineering Science), 2016, 50(4): 641-649.
[13] ZHU Guang-ming, JIANG Rong-xin, ZHOU Fan, TIAN Xiang, CHEN Yao-wu. Robust Kalman filtering algorithm with estimation of measurement biases[J]. Journal of ZheJiang University (Engineering Science), 2015, 49(7): 1343-1349.
[14] MA Teng, ZHAO Xing zhong, GAO Bo qing, WU Hui. Combined shape and topology optimization of free form structure[J]. Journal of ZheJiang University (Engineering Science), 2015, 49(10): 1946-1951.
[15] ZHANG Cheng, LI Zhi-an, GAO Bo-qing, Dong Shi-lin. Robustness analysis of reticular shells based on H∞ theory[J]. Journal of ZheJiang University (Engineering Science), 2013, 47(5): 818-823.