Please wait a minute...
Journal of ZheJiang University (Engineering Science)  2025, Vol. 59 Issue (5): 982-994    DOI: 10.3785/j.issn.1008-973X.2025.05.012
    
EWT-LSTM based industrial robot joint anomaly detection
Qincheng JIANG1(),Jianfeng TAO1,2,*(),Yangyang WANG1,Yulei ZHANG1,2,Chengliang LIU1,2
1. School of Mechanical and Power Engineering, Shanghai Jiao Tong University, Shanghai 200240, China
2. State Key Laboratory of Mechanical System and Vibration, Shanghai Jiao Tong University, Shanghai 200240, China
Download: HTML     PDF(3316KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

A novel detection method combining empirical wavelet transform (EWT) with long short-term memory (LSTM) network was proposed in order to address the challenges of joint servo parameter anomaly detection in industrial robot factory inspection scenarios, particularly under the requirements of sensor-free implementation, high accuracy and real-time performance. An integrated PLC-intelligent terminal-cloud server platform was developed for real-time joint data acquisition, enabling sensorless collection, storage and transmission of joint current and speed signals while supporting cloud-based condition monitoring and anomaly detection. EWT was employed to decompose current signals into inherently smooth characteristic components, which were directly fed into the LSTM network as input features to enhance prediction accuracy. A bidirectional LSTM architecture enhanced by convolutional neural networks (CNNs) and attention mechanisms was implemented to reconstruct complete periodic signals in order to resolve incomplete signal cycles during robotic motion. Then these reconstructed signals were compared with standard signal components through difference metrics to achieve real-time anomaly detection. Experimental validation using 1 set of standard servo parameters and 24 sets of abnormal parameters demonstrated that the proposed method achieved precise fault joint localization, maintained strong consistency with injected anomaly severity levels, and reached a detection accuracy exceeding 90%.



Key wordsindustrial robot      empirical wavelet transform      long short-term memory network      anomaly detection      cloud-edge collaboration     
Received: 01 June 2024      Published: 25 April 2025
CLC:  TP 242  
Fund:  上海市人工智能重大专项资助项目(2021SHZDZX0102).
Corresponding Authors: Jianfeng TAO     E-mail: jqc9837@sjtu.edu.cn;jftao@sjtu.edu.cn
Cite this article:

Qincheng JIANG,Jianfeng TAO,Yangyang WANG,Yulei ZHANG,Chengliang LIU. EWT-LSTM based industrial robot joint anomaly detection. Journal of ZheJiang University (Engineering Science), 2025, 59(5): 982-994.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2025.05.012     OR     https://www.zjujournals.com/eng/Y2025/V59/I5/982


基于EWT-LSTM的工业机器人关节异常检测

针对工业机器人制造企业工业机器人出厂检测场景关节伺服参数异常检测问题和不添加传感器、高准确率和实时性的需求,提出基于经验小波变换(EWT)和长短时记忆网络(LSTM)的检测方法. 构建工业机器人可编程逻辑控制器-智能终端-云服务器一体化关节实时数据采集平台,无须额外添加传感器即可实现关节电流和速度信号的实时采集、存储和传输,在云端进行状态监测和异常检测. 利用EWT分解电流信号以获得特征分量,将光滑的特征分量作为LSTM模型的输入,提高了预测准确性. 针对机器人运动周期中实时信号周期不完整的问题,采用卷积神经网络和注意力机制优化的双向LSTM模型预测补全完整的周期信号,与标准信号特征分量进行差异度量,实现实时异常检测. 采用1组标准伺服参数和24组异常伺服参数进行故障注入实验,验证了利用该方法能够定位异常关节,与注入异常程度有较好的一致性,检测准确率超过90%.


关键词: 工业机器人,  经验小波变换,  长短时记忆网络,  异常检测,  云边协同 
Fig.1 Joint motion control system of industrial robot
Fig.2 Basic structure of LSTM unit
Fig.3 Framework of EWT-LSTM anomaly detection method
Fig.4 Framework of industrial robot real-time data collection platform
Fig.5 Process of standard signal preprocessing
Fig.6 Architecture of CNN-LSTM-Attention model
层编号描述详细信息输出尺寸
Layer 1InputInput_shape = (12,6)6×12
Layer 2Conv1Dfilters=128, kernel_size=1, activation=ReLU, padding='same'128×12
Layer 3AveragePooling1Dpool_size=4128×3
Layer 4ZeroPadding1Dpadding=(1, 0)128×4
Layer 5Conv1Dfilters=128, kernel_size=1, activation=ReLU, padding='same'128×4
Layer 6AveragePooling1Dpool_size=4128×1
Layer 7Dropout0.2128×1
Layer 8BidirectionalUnits=32, return_sequences=True64×1
Layer 9Dropout0.264×1
Layer 10Permute(2,1)1×64
Layer 11Denselayer_size = 64, activation='softmax'1×64
Layer 12Permute(2,1)64×1
Layer 13Concatenate[Layer 9, Layer 12]128×1
Layer 14Flatten128
Layer 15Denselayer_size = 6, activation='sigmoid'6
Tab.1 Detail of proposed CNN-LSTM-Attention model
Fig.7 Physical picture of industrial robot joint abnormality detection test bench
设备名称型号配置
服务器CPU:Intel(R) Core(TM) i7-7700
CPU @ 3.60 GHz
内存:23.86 GB
硬盘:1.13 TB
系统:CentOS7.4
智能终端型号:NanoPi R6S
主控处理器:瑞芯微RK3588S
内存:8 GB
系统:Ubuntu 22.04
Tab.2 Hardware architecture of industrial robot joint anomaly detection test bench
关节编号伺服参数关节编号伺服参数
1Kp = 40, Ti = 504Kp = 60, Ti = 30
2Kp = 40, Ti = 505Kp = 50, Ti = 50
3Kp = 70, Ti = 506Kp = 70, Ti = 20
Tab.3 Standard servo parameter for each joint of industrial robot
实验组编号关节编号伺服参数
11Kp = 10, Ti = 50
21Kp = 25, Ti = 50
31Kp = 55, Ti = 50
41Kp = 70, Ti = 50
52Kp = 10, Ti = 50
62Kp = 25, Ti = 50
72Kp = 55, Ti = 50
82Kp = 70, Ti = 50
93Kp = 10, Ti = 50
103Kp = 25, Ti = 50
113Kp = 40, Ti = 50
123Kp = 55, Ti = 50
134Kp = 10, Ti = 30
144Kp = 25, Ti = 30
154Kp = 40, Ti = 30
164Kp = 70, Ti = 30
175Kp = 10, Ti = 50
185Kp = 25, Ti = 50
195Kp = 60, Ti = 50
205Kp = 70, Ti = 50
216Kp = 10, Ti = 20
226Kp = 25, Ti = 20
236Kp = 40, Ti = 20
246Kp = 55, Ti = 20
Tab.4 Servo parameter setting of experimental group for abnormal joint of industrial robot
Fig.8 Frequency domain preprocessing of joint 1 standard signal
Fig.9 Extraction result of characteristic component of joint 1 standard signal
Fig.10 Training and validation loss during model training process
模型MAERMSER2
RF0.053470.059250.87383
SVR0.040510.060910.87467
RNN0.042940.046300.91125
CNN0.035970.040880.93080
LSTM0.031640.036220.95399
seq2seq0.018830.022670.98197
LSTM-Attention0.014540.018630.98782
seq2seq-Attention0.012920.016910.98997
CNN-LSTM-Attention0.006910.008530.99662
Tab.5 Evaluation result of industrial robot joint signal predicted by time series prediction model
Fig.11 Comparison of prediction result for different input of model joint 1
Fig.12 Abnormal detection result of six joints of industrial robot obtained by three abnormal detection methods
关节编号D
相关系数法峰值差异法均方差法
11.12×10?73.656.21×10?4
28.35×10?9108.301.19×10?6
33.77×10?93.101.97×10?2
45.94×10?714.241.00×10?1
51.28×10?81.629.96×10?3
63.29×10?85.261.39×10?3
Tab.6 Variance of abnormal detection result of each joint of industrial robot in 3 cycles
[1]   YAMADA Y, NAGAMATSU S, SATO Y. Development of multi-arm robots for automobile assembly [C]// Proceedings of IEEE International Conference on Robotics and Automation . Nagoya: IEEE, 1995: 2224-2229.
[2]   张康. 工业机器人柔性关节关键子部件健康评估方法研究[D]. 上海: 上海交通大学, 2022.
ZHANG Kang. Research on health assessment method of key components of flexible joint of industrial robot [D]. Shanghai: Shanghai Jiao Tong University, 2022.
[3]   SPIEWAK S A, DUGGIRALA R, BARNETT K Predictive monitoring and control of the cold extrusion process[J]. CIRP Annals, 2000, 49 (1): 383- 386
doi: 10.1016/S0007-8506(07)62970-9
[4]   YANG Bin, GUO Kai, ZHOU Qian, et al Early chatter detection in robotic milling under variable robot postures and cutting parameters[J]. Mechanical Systems and Signal Processing, 2023, 186: 109860
doi: 10.1016/j.ymssp.2022.109860
[5]   IZAGIRRE U, ANDONEGUI I, ECIOLAZA L, et al Towards manufacturing robotics accuracy degradation assessment: a vision-based data-driven implementation[J]. Robotics and Computer-Integrated Manufacturing, 2021, 67: 102029
doi: 10.1016/j.rcim.2020.102029
[6]   HE Ruikai, XU Peng, CHEN Zhibo, et al A non-intrusive approach for fault detection and diagnosis of water distribution systems based on image sensors, audio sensors and an inspection robot[J]. Energy and Buildings, 2021, 243: 110967
doi: 10.1016/j.enbuild.2021.110967
[7]   CHEN Tingting, LIU Xueping, XIA Bizhong, et al Unsupervised anomaly detection of industrial robots using sliding-window convolutional variational autoencoder[J]. IEEE Access, 2020, 8: 47072- 47081
doi: 10.1109/ACCESS.2020.2977892
[8]   MUHAMMET O, NEIL D S, ERDEM O. Robotically assisted active vibration control in milling: a feasibility study[J]. Mechanical Systems and Signal Processing, 2022, 177: 109152
[9]   QIAO Guixiu, WEISS B A Quick health assessment for industrial robot health degradation and the supporting advanced sensing development[J]. Journal of Manufacturing Systems, 2018, 48: 51- 59
doi: 10.1016/j.jmsy.2018.04.004
[10]   潘京辉. 多关节协作机器人故障诊断及容错控制研究[D]. 北京: 北京科技大学, 2023.
PAN Jinghui. Fault diagnosis and fault tolerant control of multi-joint cooperative robot [D]. Beijing: University of Science and Technology Beijing, 2023.
[11]   GAJULA K, MAREPALLI L K, YAO Xiu, et al Recursive least squares and adaptive Kalman filter-based state and parameter estimation for series arc fault detection on DC microgrids[J]. IEEE Journal of Emerging and Selected Topics in Power Electronics, 2022, 10 (4): 4715- 4724
doi: 10.1109/JESTPE.2021.3135409
[12]   XIANG Xianbo, YU Caoyang, ZHANG Qin On intelligent risk analysis and critical decision of underwater robotic vehicle[J]. Ocean Engineering, 2017, 140: 453- 465
doi: 10.1016/j.oceaneng.2017.06.020
[13]   LIU Yutian, CHEN Jungan. Integrated fault diagnosis method of mobile robot [C]// Theoretical and Mathematical Foundations of Computer Science . Berlin: Springer, 2011: 372-379.
[14]   WU You, FU Zhuang, FEI Jian Fault diagnosis for industrial robots based on a combined approach of manifold learning, treelet transform and Naive Bayes[J]. Review of Scientific Instruments, 2020, 91 (1): 015116
doi: 10.1063/1.5118000
[15]   FAN Song, ZHANG Limin, WANG Yufeng, et al. Vision-based fault classification for monitoring industrial robot [C]// 37th Chinese Control Conference . Wuhan: IEEE, 2018: 5889-5894.
[16]   LI Jimeng, YAO Xifeng, WANG Xiangdong, et al Multiscale local features learning based on BP neural network for rolling bearing intelligent fault diagnosis[J]. Measurement, 2020, 153: 107419
doi: 10.1016/j.measurement.2019.107419
[17]   ELANGOVAN K, YOKH K T, MOHAN R E, et al Fault diagnosis of a reconfigurable crawling–rolling robot based on support vector machines[J]. Applied Sciences, 2017, 7 (10): 1025
doi: 10.3390/app7101025
[18]   马百腾. 基于EEMD-HT-峭度分析的工业机器人关节电流信号异常检测方法: 2022100920200 [P]. 2022-01-26.
[19]   CHENG Fangzhou, RAGHAVAN A, JUNG D, et al. High-accuracy unsupervised fault detection of industrial robots using current signal analysis [C]// IEEE International Conference on Prognostics and Health Management . San Francisco: IEEE, 2019: 1–8.
[20]   赵阳. 基于改进经验小波变换的风电机组齿轮箱故障诊断[D]. 北京: 北京交通大学, 2020.
ZHAO Yang. Fault diagnosis of wind turbine gearbox based on improved empirical wavelet transform [D]. Beijing: Beijing Jiao Tong University, 2020.
[21]   XU Yonggang, ZHANG Kun, MA Chaoyong, et al An improved empirical wavelet transform and its applications in rolling bearing fault diagnosis[J]. Applied Sciences, 2018, 8 (12): 2352
doi: 10.3390/app8122352
[22]   庞学丽, 宋坤, 姚红云, 等 基于VMD-ISSA-LSTM的短时交通流预测研究[J]. 现代电子技术, 2024, 47 (8): 31- 36
PANG Xueli, SONG Kun, YAO Hongyun, et al Research on short-term traffic flow prediction based on VMD-ISSA-LSTM[J]. Modern Electronics Technique, 2024, 47 (8): 31- 36
[23]   高正. 基于EWT和特征融合的钻机钻杆故障识别研究[D]. 杭州: 浙江大学, 2019.
GAO Zheng. Research on fault identification of drill pipe drill based on EWT and feature fusion [D]. Hangzhou: Zhejiang University, 2019.
[24]   GILLES J Empirical wavelet transform[J]. IEEE Transactions on Signal Processing, 2013, 61 (16): 3999- 4010
doi: 10.1109/TSP.2013.2265222
[25]   龙霞飞, 何志成, 周凌, 等. 基于KOA-CNN-BiLSTM-AM的风电齿轮箱故障诊断研究[EB/OL]. [2025-02-09]. http://kns.cnki.net/kcms/detail/44.1259.TH.20240506.1728.002.html.
[26]   马良玉, 吕若萌 用SSA优化CNN-LSTM-SEnet预测模型实现风电机组故障预警[J]. 电力科学与工程, 2024, 40 (6): 1- 10
MA Liangyu, LV Ruomeng Wind turbine fault warning with SSA optimized CNN-LSTM-SEnet prediction model[J]. Electric Power Science and Engineering, 2024, 40 (6): 1- 10
doi: 10.3969/j.ISSN.1672-0792.2024.06.001
[27]   BAHDANAU D, CHO K, BENGIO Y. Neural machine translation by jointly learning to align and translate: arXiv: 1409.0473 [EB/OL]. [2024-12-20]. https://formacion.actuarios.org/wp-content/uploads/2024/05/1409.0473-Neural-Machine-Translation-By-Jointly-Learning-To-Align-And-Translate.pdf.
[28]   唐宇翔, 陶建峰, 刘成良 盾构机刀盘主驱动电机异常检测与性能评估[J]. 西安交通大学学报, 2023, 57 (10): 132- 142
TANG Yuxiang, TAO Jianfeng, LIU Chengliang Abnormal detection and performance evaluation of main drive motor of shield tunneling machine cutter head[J]. Journal of Xi'an Jiaotong University, 2023, 57 (10): 132- 142
doi: 10.7652/xjtuxb202310013
[29]   杨琦, 陈智才 基于EMD和相关系数法的列车滚动轴承故障诊断方法研究[J]. 电力机车与城轨车辆, 2018, 41 (3): 15- 17
YANG Qi, CHEN Zhicai Fault diagnosis method for train rolling bearing based on EMD and CCM[J]. Electric Locomotives and Mass Transit Vehicles, 2018, 41 (3): 15- 17
[1] Bote WANG,Qing WANG,Qiang LIU,Bo JIN. Self-supervised anomaly recognition method for wind turbine blade based on multi-channel vibration principal features[J]. Journal of ZheJiang University (Engineering Science), 2025, 59(3): 653-660.
[2] Yunlong ZHAO,Minzhe ZHAO,Wenqiang ZHU,Xingyu CHA. Cloud-edge collaborative natural language processing method based on lightweight transfer learning[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(12): 2531-2539.
[3] Wan-jin GUO,Wu-duan ZHAO,Su-yang YU,Li-jun ZHAO,Chu-qing CAO. Active adaptive online trajectory prediction for robotic grinding on surface without prior model[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(8): 1655-1666.
[4] Xue-qi XING,Yu-tong DING,Tang-bin XIA,Er-shun PAN,Li-feng XI. Integrated modeling of commercial aircraft maintenance plan recommendation system based on knowledge graph[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(3): 512-521.
[5] Wan-jin GUO,Wu-duan ZHAO,Qian-hui LI,Li-jun ZHAO,Chu-qing CAO. Ensemble probabilistic model based variable impedance for robotic grinding force control[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(12): 2356-2366.
[6] Fei SUN,Xiao-run LI,Liao-ying ZHAO,Shao-qi YU. Anomaly detection algorithm based on FrFT transform and total variation regularization[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(7): 1276-1284.
[7] Lin TONG,Zheng GUAN,Li-wei WANG,Wen-tao YANG,Yang YAO. New energy ramp event prediction based on time series decomposition and error correction[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(2): 338-346.
[8] Peng ZHANG,Zi-du TIAN,Hao WANG. Flight parameter data anomaly detection method based on improved generative adversarial network[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(10): 1967-1976.
[9] Hao-ran MA,Ya-bin DING. Calibration method of laser displacement sensor based on binocular vision[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(9): 1634-1642.
[10] Wei-qi CHEN,Jing-chang WANG,Ling CHEN,Yong-qin YANG,Yong WU. Prediction model of multi-factor aware mobile terminal replacement based on deep neural network[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(1): 109-115.
[11] Da-zhao DONG,Guan-hua XU,Ji-liang GAO,Yue-tong XU,Jian-zhong FU. Online correction algorithm for posture by robot assembly based on machine vision[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(1): 145-152.
[12] Wen-shu LI,Tao-tao ZOU,Hong-yan WANG,Hai HUANG. Traffic accident quantity prediction model based on dual-scale long short-term memory network[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(8): 1613-1619.
[13] Yi-xiong FENG,Kang-jie LI,Yi-cong GAO,Hao ZHENG. Corner recognition of industrial robot contour curve for visual servoing[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(8): 1449-1456.
[14] Ying-jie GUO,Fan GU,Hui-yue DONG,Hai-jin WANG. Prediction and compensation of robot deformation under pressure force of pressure foot[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(8): 1457-1465.
[15] Chen-tao MAO,Zhang-wei CHEN,Xiang ZHANG,Hong-fei ZU. Kinematic calibration for robots based on relative accuracy[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(7): 1316-1324.