Please wait a minute...
Journal of ZheJiang University (Engineering Science)  2026, Vol. 60 Issue (3): 536-545    DOI: 10.3785/j.issn.1008-973X.2026.03.009
    
Object recognition of excavator operation based on improved CNN-LSTM
Congyu HU1(),Chenbo YIN1,*(),Wei MA1,Chao YANG1,Shikuan YAN2
1. College of Mechanical and Power Engineering, Nanjing Tech University, Nanjing 211816, China
2. Jiangsu Tianzhou Testing Technology Limited Company, Nanjing 210019, China
Download: HTML     PDF(3424KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

A hybrid neural network model (HBA-CNN-LSTM-Attention) was proposed for excavator operation object recognition in order to address the limitations of conventional excavator operation object recognition methods, which often suffered from complex computational models and high sensitivity to lighting conditions in practical applications. The strength of convolutional neural network (CNN) in spatial feature extraction was integrated and the squeeze-and-excitation (SE) attention mechanism was incorporated to adaptively optimize feature channel weights, thereby enhancing the effectiveness of feature representation. The model attained greater stability when dealing with dynamic operating scenarios by exploiting the modeling capability of long short-term memory (LSTM) network for temporal information. The maximum information coefficient (MIC) was employed to select input feature in order to further improve performance, highlighting the critical feature. The honey badger algorithm (HBA) was used to automatically tune the hyperparameters of the network, ensuring an optimal configuration during training. Experiments conducted on a representative operational dataset showed that the proposed model achieved a classification accuracy of 96.61%, representing a significant improvement over conventional model. Results verified the effectiveness and practicality of the hybrid model in complex operating environment.



Key wordsexcavator      operation object      combination neural network      feature selection      honey badger algorithm (HBA)      attention mechanism     
Received: 30 June 2025      Published: 04 February 2026
CLC:  TP 181  
Fund:  国家重点研发计划资助项目(2021YFB2011904).
Corresponding Authors: Chenbo YIN     E-mail: hucongyu_iacm@163.com;yinchenbo@njtech.edu.cn
Cite this article:

Congyu HU,Chenbo YIN,Wei MA,Chao YANG,Shikuan YAN. Object recognition of excavator operation based on improved CNN-LSTM. Journal of ZheJiang University (Engineering Science), 2026, 60(3): 536-545.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2026.03.009     OR     https://www.zjujournals.com/eng/Y2026/V60/I3/536


基于改进CNN-LSTM的挖掘机作业对象识别

针对传统挖掘机作业对象识别方法因计算模型复杂、易受光照条件影响,在实际应用中表现出明显局限性的问题,提出组合神经网络模型(HBA-CNN-LSTM-Attention),用于挖掘机作业对象识别. 该模型融合卷积神经网络(CNN)在提取空间特征方面的优势,引入压缩与激励(SE)注意力机制,实现对特征通道权重的自适应优化,提高了特征表达的有效性. 借助长短期记忆网络(LSTM)在处理时序信息中的建模能力,使模型在应对动态作业场景时更具稳定性. 为了进一步增强模型性能,采用最大信息系数(MIC)对输入特征进行筛选,以突出关键特征. 利用蜜獾优化算法(HBA)自动调整网络结构中的超参数,确保模型在训练过程中达到最优配置. 在典型作业数据集上的实验显示,该模型在分类任务中取得了96.61%的准确率,较传统模型在精度上有显著的提升. 实验结果验证了组合模型在复杂作业环境下的有效性与实用性.


关键词: 挖掘机,  作业对象,  组合神经网络,  特征选择,  蜜獾优化算法(HBA),  注意力机制 
Fig.1 Typical operation object and data acquisition equipment
Fig.2 Data collection cloud platform
设备型号主要参数
液压挖掘机SY980H574/2 100 kW/(r·min?1)
传感器HG3350G10NMPXM10~35 MPa/0.1%FS
控制器BODAS RC28-14/3050~250 Hz
CAN总线分析仪USBCAN-Ⅱ Pro50~8 000 kb/s
数据记录仪ZLG CANDTU-200UR5~1 000 kb/s
Tab.1 Key equipment and main parameter of excavator experimental platform
Fig.3 Hydraulic schematic diagram of excavator
特征特征变量
X1发动机转速/(r·min?1)
X2冷却液温度/℃
X3液压油温度/℃
X4铲斗挖掘先导压力/Pa
X5铲斗挖掘先导电压/mV
X6铲斗卸载先导压力/Pa
X7铲斗卸载先导电压/mV
X8铲斗大腔压力/Pa
X9斗杆挖掘先导压力/Pa
X10斗杆卸载先导压力/Pa
X11斗杆挖掘先导电压/mV
X12斗杆卸载先导电压/mV
X13斗杆大腔压力/Pa
X14斗杆小腔压力/Pa
X15动臂大腔压力/Pa
X16动臂小腔压力/Pa
X17前泵压力/Pa
X18后泵压力/Pa
Tab.2 Characteristic of preliminary screening
特征特征变量
X2冷却液温度/℃
X3液压油温度/℃
X5铲斗挖掘先导电压/mV
X7铲斗卸载先导电压/mV
X8铲斗大腔压力/Pa
X11斗杆挖掘先导电压/mV
X12斗杆卸载先导电压/mV
X13斗杆大腔压力/Pa
X14斗杆小腔压力/Pa
X15动臂大腔压力/Pa
X17前泵压力/Pa
X18后泵压力/Pa
Tab.3 Feature selected based on MIC
Fig.4 Autocorrelation function of original data
Fig.5 Noise probability density function graph
Fig.6 CNN network structure
Fig.7 Structure of LSTM network
Fig.8 Structure of CNN-LSTM-Attention network
超参数数值
隐含层神经元数量 Nh[16, 128]
神经元丢弃率 d[0.1, 0.6]
初始学习率 r[1×10?6, 1×10?2]
L2正则化参数 CL[1×10?10, 1×10?2]
Tab.4 Hyper parameter of HBA
Fig.9 Classification result of different operation object
模型NhdrCL
CNN320.20.0010.0001
HBA-CNN730.317190.003386.21×10?5
GRU320.20.0010.0001
HBA-GRU660.223000.007074.04×10?9
LSTM320.20.0010.0001
HBA-LSTM430.117540.001279.47×10?8
CNN-LSTM-Attention320.20.0010.0001
HBA-CNN-LSTM-Attention760.475930.001534.37×10?6
Tab.5 Hyper parameter setting of different models
模型APRF1
CNN0.917950.915980.918080.92930
GRU0.838700.841600.842450.83803
LSTM0.939880.940080.943850.94215
CNN-LSTM-Attention0.966100.963230.963750.96335
Tab.6 Classification performance of HBA optimized model
模型ta/sts/s
CNN1.610.0107
HBA-CNN1.720.0115
GRU2.190.0146
HBA-GRU2.330.0155
LSTM1.750.0117
HBA-LSTM1.870.0125
CNN-LSTM-Attention3.970.0265
HBA-CNN-LSTM-Attention4.130.0275
Tab.7 Real-time prediction time of each model
模型特征集APRF1
HBA-CNNS10.93080.93980.93830.9384
S20.92870.92230.92250.9224
S30.93250.92270.93230.9373
S40.87980.87910.87920.8896
HBA-GRUS10.87630.88770.89190.8762
S20.86210.86160.86200.8613
S30.87190.87160.87170.8712
S40.74430.74530.74420.7434
HBA-LSTMS10.95020.95600.95710.9566
S20.93300.93320.93470.9339
S30.94740.94070.94840.9445
S40.92890.93040.93520.9336
HBA-CNN-
LSTM-Attention
S10.98210.98110.98130.9812
S20.97840.97860.97730.9775
S30.97960.97810.97620.9771
S40.92430.91510.92020.9176
Tab.8 Performance indicator of different models on each feature set
[1]   JIN Z, PAGILLA P, MASKE H, et al Task learning, intent prediction, and adaptive blended shared control with application to excavators[J]. IEEE Transactions on Control Systems Technology, 2021, 29 (1): 18- 28
doi: 10.1109/TCST.2019.2959536
[2]   陆亮, 吴军凯, 孙宁, 等 智能建造: 工程机械智能化[J]. 液压与气动, 2022, 46 (6): 1- 9
LU Liang, WU Junkai, SUN Ning, et al Intelligent construction: construction machinery intelligentization[J]. Chinese Hydraulics and Pneumatics, 2022, 46 (6): 1- 9
doi: 10.11832/j.issn.1000-4858.2022.06.001
[3]   李运华, 范茹军, 杨丽曼, 等 智能化挖掘机的研究现状与发展趋势[J]. 机械工程学报, 2020, 56 (13): 165- 178
LI Yunhua, FAN Rujun, YANG Liman, et al Research status and development trend of intelligent excavators[J]. Journal of Mechanical Engineering, 2020, 56 (13): 165- 178
doi: 10.3901/JME.2020.13.165
[4]   赵圣奎, 侯向泽, 王波 基于模糊控制算法的露天铁矿挖掘机控制研究[J]. 自动化应用, 2024, (24): 91- 93
ZHAO Shengkui, HOU Xiangze, WANG Bo Research on control of open-pit iron mine excavator based on fuzzy control algorithm[J]. Automation Application, 2024, (24): 91- 93
[5]   DOBSON A, MARSHALL J, LARSSON J Admittance control for robotic loading: design and experiments with a 1-tonne loader and a 14-tonne load-haul-dump machine[J]. Journal of Field Robotics, 2017, 34 (1): 123- 150
doi: 10.1002/rob.21654
[6]   MOGHADDAM R Y, KOTCHON A, LIPSETT M G Method and apparatus for on-line estimation of soil parameters during excavation[J]. Journal of Terramechanics, 2012, 49 (3/4): 173- 181
[7]   ALTHOEFER K, TAN C P, ZWEIRI Y H, et al Hybrid soil parameter measurement and estimation scheme for excavation automation[J]. IEEE Transactions on Instrumentation and Measurement, 2009, 58 (10): 3633- 3641
doi: 10.1109/TIM.2009.2018699
[8]   LI J, CHEN C, LI Y, et al Difficulty assessment of shoveling stacked materials based on the fusion of neural network and radar chart information[J]. Automation in Construction, 2021, 132: 103966
doi: 10.1016/j.autcon.2021.103966
[9]   SARATA S, KOYACHI N, SUGAWARA K. Field test of autonomous loading operation by wheel loader [C]//Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems. Nice: IEEE, 2008: 2661–2666.
[10]   FERNANDO H, MARSHALL J What lies beneath: material classification for autonomous excavators using proprioceptive force sensing and machine learning[J]. Automation in Construction, 2020, 119: 103374
doi: 10.1016/j.autcon.2020.103374
[11]   LI S, WANG S, CHEN X, et al Application of physics-informed machine learning for excavator working resistance modeling[J]. Mechanical Systems and Signal Processing, 2024, 209: 111117
doi: 10.1016/j.ymssp.2024.111117
[12]   YUAN G, LI X, QIU P, et al Feature selection method based on wavelet similarity combined with maximum information coefficient[J]. Information Sciences, 2025, 699: 121801
doi: 10.1016/j.ins.2024.121801
[13]   王斐, 梁晓庚, 王彦奎, 等 增强小波系数的飞行数据奇异值阈值降噪[J]. 火力与指挥控制, 2013, 38 (7): 171- 173
WANG Fei, LIANG Xiaogeng, WANG Yankui, et al Flight data de-noising using enhanced wavelet coefficients and threshold shrinkage in wavelet transform with singular value decomposition[J]. Fire Control and Command Control, 2013, 38 (7): 171- 173
[14]   刘旭宙, 秦满忠, 郭晓, 等. 用噪声概率密度函数对比地震计观测性能 [C]//2018年中国地球科学联合学术年会论文集(二十四)——专题48: 环境地球物理技术应用与研究进展、专题49: 浅地表地球物理进展. 北京: [s. n. ], 2018: 64–67.
LIU Xuzhou, QIN Manzhong, GUO Xiao, et al. Comparison of seismometer observation performance with noise probability density function [C]// 2018 China Geosciences Joint Academic Annual Meeting (Vol. 24)—Session 48: Advances in Environmental Geophysical Technology Applications and Exploration; Session 49: Advances in Near-Surface Geophysics. Beijing: [s. n.], 2018: 64−67.
[15]   WU J, HAO Z, WANG S, et al Nonlinear comb narrow-band noise removal using unscented Kalman filter: feasibility and field test analysis[J]. Measurement, 2025, 242: 115928
doi: 10.1016/j.measurement.2024.115928
[16]   KOLARIK M, BURGET R, RIHA K. Comparing normalization methods for limited batch size segmentation neural networks [C]//Proceedings of the 43rd International Conference on Telecommunications and Signal Processing. Milan: IEEE, 2020: 677-680.
[17]   WANG Y, HONG K, ZOU J, et al A CNN-based visual sorting system with cloud-edge computing for flexible manufacturing systems[J]. IEEE Transactions on Industrial Informatics, 2020, 16 (7): 4726- 4735
doi: 10.1109/TII.2019.2947539
[18]   CHAKRABORTY S, BANIK J, ADDHYA S, et al. Study of dependency on number of LSTM units for character based text generation models [C]//Proceedings of the International Conference on Computer Science, Engineering and Applications. Gunupur: IEEE, 2020: 1–5.
[1] Wenqiang CHEN,Linyue FENG,Dongdan WANG,Yulei GU,Xuan ZHAO. Vehicle trajectory prediction model integrating dynamic risk map and multivariate attention mechanism[J]. Journal of ZheJiang University (Engineering Science), 2026, 60(3): 455-467.
[2] Shuang WANG,Xitai ZHANG,Yongcun GUO,Shousuo SUN. Demagnetization fault diagnosis of controllable hybrid magnetic couplers based on deep neural networks[J]. Journal of ZheJiang University (Engineering Science), 2026, 60(2): 279-286.
[3] Xianhua LI,Pengfei DU,Tao SONG,Xun QIU,Yu CAI. EEG signal classification based on multi-scale sliding-window attention temporal convolutional networks[J]. Journal of ZheJiang University (Engineering Science), 2026, 60(2): 370-378.
[4] Minghui YANG,Muyuan SONG,Daxi FU,Yanwei GUO,Xianzhui LU,Wencong ZHANG,Weilong ZHENG. Prediction of shield tunneling-induced soil settlement based on multi-head self-attention-Bi-LSTM model[J]. Journal of ZheJiang University (Engineering Science), 2026, 60(2): 415-424.
[5] Siyao ZHOU,Nan XIA,Jiahong JIANG. Pose-guided dual-branch network for clothing-changing person re-identification[J]. Journal of ZheJiang University (Engineering Science), 2026, 60(1): 71-80.
[6] Fujian WANG,Zetian ZHANG,Xiqun CHEN,Dianhai WANG. Usage prediction of shared bike based on multi-channel graph aggregation attention mechanism[J]. Journal of ZheJiang University (Engineering Science), 2025, 59(9): 1986-1995.
[7] Xuejun ZHANG,Shubin LIANG,Wanrong BAI,Fenghe ZHANG,Haiyan HUANG,Meifeng GUO,Zhuo CHEN. Source code vulnerability detection method based on heterogeneous graph representation[J]. Journal of ZheJiang University (Engineering Science), 2025, 59(8): 1644-1652.
[8] Yishan LIN,Jing ZUO,Shuhua LU. Multimodal sentiment analysis based on multi-head self-attention mechanism and MLP-Interactor[J]. Journal of ZheJiang University (Engineering Science), 2025, 59(8): 1653-1661.
[9] Yahong ZHAI,Yaling CHEN,Longyan XU,Yu GONG. Improved YOLOv8s lightweight small target detection algorithm of UAV aerial image[J]. Journal of ZheJiang University (Engineering Science), 2025, 59(8): 1708-1717.
[10] Jiarui FU,Zhaofei LI,Hao ZHOU,Wei HUANG. Camouflaged object detection based on Convnextv2 and texture-edge guidance[J]. Journal of ZheJiang University (Engineering Science), 2025, 59(8): 1718-1726.
[11] Rongtai YANG,Yubin SHAO,Qingzhi DU. Structure-aware model for few-shot knowledge completion[J]. Journal of ZheJiang University (Engineering Science), 2025, 59(7): 1394-1402.
[12] Shengju WANG,Zan ZHANG. Missing value imputation algorithm based on accelerated diffusion model[J]. Journal of ZheJiang University (Engineering Science), 2025, 59(7): 1471-1480.
[13] Yongqing CAI,Cheng HAN,Wei QUAN,Wudi CHEN. Visual induced motion sickness estimation model based on attention mechanism[J]. Journal of ZheJiang University (Engineering Science), 2025, 59(6): 1110-1118.
[14] Wenbo JU,Huajun DONG. Motherboard defect detection method based on context information fusion and dynamic sampling[J]. Journal of ZheJiang University (Engineering Science), 2025, 59(6): 1159-1168.
[15] Xiangyu ZHOU,Yizhi LIU,Yijiang ZHAO,Zhuhua LIAO,Decheng ZHANG. Hierarchical spatial embedding BiGRU model for destination prediction[J]. Journal of ZheJiang University (Engineering Science), 2025, 59(6): 1211-1218.