Please wait a minute...
浙江大学学报(工学版)
Journal of ZheJiang University (Engineering Science)  2022, Vol. 56 Issue (11): 2109-2118    DOI: 10.3785/j.issn.1008-973X.2022.11.001
    
Remaining useful life estimation of turbofan engine based on selective ensemble of deep neural networks
Dong-yang HAN1,2(),Ze-yu LIN1,Yu ZHENG1,Mei-mei ZHENG1,Tang-bin XIA1,2,*()
1. School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China
2. Fraunhofer Project Center for Smart Manufacturing, Shanghai Jiao Tong University, Shanghai 201306, China
Download: HTML     PDF(1516KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

The large sample and high-dimensional condition monitoring data has technical challenges for accurate prediction of remaining useful life (RUL). A two-stage selective deep neural network ensemble method based on high-dimensional data was proposed to improve the prediction accuracy and convergence efficiency of complex equipments represented by aviation turbofan engine. The first stage was the candidate set generation scheme under the joint disturbance of multiple methods. By using heterogeneous neural network structure, multi time scale design and algorithm parameter randomization to eliminate the internal coupling relationship of the model, the diversity of candidate deep neural network sets was enhanced. In the second stage, the genetic algorithm was used to integrate pruning redundant models. The method effectively the eliminated redundant learners with poor performance to obtain the diversified optimal candidate subsets and output the prediction results according to the average integration. The final result was estimated by average. The experimental comparison with the data of the individual model shows that the proposed method enhances the accuracy and diversity of the ensemble model and improves the RUL prediction result by nearly 20%, which provides stable support for maintenance decision.

Key words： remaining useful life      selective ensemble learning      heterogeneous deep neural network      ensemble diversity      turbofan engine     
Received: 16 February 2022      Published: 02 December 2022
CLC:  TH183  
Fund:  国家自然科学基金资助项目(51875359)；教育部-中国移动联合基金建设项目(MCM20180703);上海市“科技创新行动计划”自然科学基金资助项目(20ZR1428600)；上海商用飞机系统工程科创中心联合研究基金资助项目(FASE-2021-M7);上海交通大学深蓝计划基金资助项目(SL2021MS008)
Corresponding Authors: Tang-bin XIA     E-mail: handongyang@sjtu.edu.cn;xtbxtb@sjtu.edu.cn
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Dong-yang HAN
Ze-yu LIN
Yu ZHENG
Mei-mei ZHENG
Tang-bin XIA
Cite this article:

Dong-yang HAN,Ze-yu LIN,Yu ZHENG,Mei-mei ZHENG,Tang-bin XIA. Remaining useful life estimation of turbofan engine based on selective ensemble of deep neural networks. Journal of ZheJiang University (Engineering Science), 2022, 56(11): 2109-2118.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2022.11.001     OR     https://www.zjujournals.com/eng/Y2022/V56/I11/2109


基于选择性深度神经网络集成的涡扇发动机剩余寿命预测

大样本高维度状态监测数据对剩余使用寿命（RUL）精准预测有着技术性挑战，为了提高以航空涡轮风扇发动机为代表的复杂装备的预测精度和收敛效率，提出一种两阶段的选择性深度神经网络集成方法. 第1阶段为多方法联合扰动下的候选集生成方案，通过采用异质神经网络结构、多时间尺度设计和算法参数随机化消除模型内部耦合关系，强化候选深度神经网络集多样性；第2阶段利用遗传算法集成修剪冗余模型，有效剔除性能不佳的冗余学习器，以获取多样化最优候选子集，并按平均集成输出预测结果. 与个体模型的数据实验对比表明，所提方法通过同步增强集成模型准确性和多样性，提升了近20%的RUL预测精度，可为运维决策提供有力支撑.


关键词： 剩余寿命预测,  选择性集成,  异质深度神经网络,  集成多样性,  涡扇发动机 
Fig.1 Framework of selective neural network ensemble model
Fig.2 Long short term memory
Fig.3 CNN-LSTM combined neural network
Fig.4 Individual chromosome setting of genetic algorithm
物理特性描述 单位 物理特性描述 单位
风扇入口总温度 兰氏度 风扇转换转速 r/min
低压压缩机温度 兰氏度 核心机转换转速 r/min
高压压缩机温度 兰氏度 抽汽焓 ?
旁路管道总压强 磅力/平方英寸 需求风扇转速 r/min
高压压缩机压强 磅力/平方英寸 旁通比 ?
低压涡轮温度 兰氏度 燃烧室油气比 ?
风扇进口压强 磅力/平方英寸 需求风扇转换转速 r/min
物理风扇转速 r/min 高压涡轮冷气流量 磅/s
物理核心机转速 r/min 低压涡轮冷气流量 磅/s
发动机压力比 ? 飞行高度 千英尺
高压压缩机静压 磅力/平方英寸 马赫数 ?
燃料流量与静压比 秒脉冲数/
（磅力/平方英寸） 		节流器角度 （°）
Tab.1 Turbine engine condition parameters
名称 训练发
动机数 		测试发
动机数 		训练样
本数 		测试样
本数 		传感
器数 		运行
参数
FD001 100 100 20 630 13 095 21 3
FD003 100 100 24 720 16 596 21 3
Tab.2 Descriptions of dataset FD001 and FD003
Fig.5 Sensor signal trend of decay, growth and nontrend
Fig.6 Schematic diagram of segmented RUL label
Fig.7 Schematic diagram of Heterogeneous neural network structure　　　　　　　　
参数 数值
批尺寸 128
训练时期数 100或达到early stopping停止条件
LSTM节点数 [20，40]
卷积核数 [50，100]
卷积核大小 [1，3]
隐蔽层节点数 30
Dropout 0.5
遗传算法初始种群大小 100
终止迭代代数 1 000
交叉概率 0.7
Tab.3 Key parameter setting of networks and GA
基模型 模型参数量 FLOPs/G
LSTM(时间窗30) 34 501 4.72×10?4
CNN-LSTM(60) 74 776 3.53×10?2
CNN(90) 317 417 3.88×10?1
CNN(120) 553 741 5.52×10?1
Tab.4 Computational complexity of base models in ensemble process
测试集名称 $ {\mathit{t}}_{\mathit{i}}\ge 120 $ $ 120 > {\mathit{t}}_{\mathit{i}}\ge 90 $ $ 90 > {\mathit{t}}_{\mathit{i}}\ge 60 $ $ 60 > {\mathit{t}}_{\mathit{i}}\ge 30 $
FD001 63 11 14 12
FD003 65 16 13 6
Tab.5 Data bulk of FD001 and FD003 test set
Fig.8 Predict value of ensemble models with different capacity
Fig.9 Convergence of selection process
Fig.10 Truth and prediction of RUL in FD001
预测方法 RMSE SCORE
FD001 FD003 FD001 FD003
Deep-CNN[6] 12.61 12.64 274.00 284.10
Deep-LSTM[25] 16.14 16.18 338.00 852.00
CNN-LSTM[26] 16.13 17.12 303.00 1 420.00
RNN-SPI[27] 13.58 19.16 228.00 1 727.00
MODBNE[15] 15.04 19.41 334.00 683.40
RULCLIPPER[28] 13.27 16.00 216.00 317.00
选择性神经网络集成 12.00 13.08 282.00 314.80
Tab.6 Comparison results with methods proposed
Fig.11 RMSE of each test subset in FD001
方法 RMSE SCORE
FD001 FD003 FD001 FD003
LSTM 14.28 16.97 362.00 674.00
CNN 13.32 14.23 324.00 350.00
CNN-LSTM[ 14.01 17.19 354.00 731.00
选择性神经网络集成 12.00 13.08 282.00 314.00
Tab.7 Comparison of prediction ability between ensemble model and individual model
方法 RMSE SCORE
FD001 FD003 FD001 FD003
无多样性方法 12.96 14.47 292.00 340.00
多扰动多样性提升 12.00 13.08 282.00 314.00
Tab.8 Comparison of results of diversity methods with or without joint perturbation
方法 RMSE SCORE
FD001 FD003 FD001 FD003
无选择过程 12.26 13.99 288.00 343.00
选择性集成 12.00 13.08 282.00 314.00
Tab.9 Comparison of results with or without ensemble process
[1]   XIA T B, DONG Y F, XIAO L, et al Recent advances in prognostics and health management for advanced manufacturing paradigms[J]. Reliability Engineering and System Safety, 2018, 178 (1): 255- 268
[2]   雷克兵, 陈自强 基于改进多新息扩展卡尔曼滤波的电池SOC估计[J]. 浙江大学学报: 工学版, 2021, 55 (10): 1978- 1985
LEI Ke-bing, CHEN Zi-qiang Estimation of state of charge of battery based on improved multi-innovation extended Kalman filter[J]. Journal of Zhejiang University: Engineering Science, 2021, 55 (10): 1978- 1985
[3]   CHEN X W, LIU Z, WANG J Y, et al An adaptive prediction model for the remaining life of an Li-Ion battery based on the fusion of the two-phase wiener process and an extreme learning machine[J]. Electronics, 2021, 10 (5): 540
doi: 10.3390/electronics10050540
[4]   CELESTINO O, FERNANDO S L, JAVIER R P, et al A hybrid ARIMA–SVM model for the study of the remaining useful life of aircraft engines[J]. Journal of Computational and Applied Mathematics, 2019, 346 (15): 184- 191
[5]   DEUTSCH J, D HE Using deep learning-based approach to predict remaining useful life of rotating components[J]. IEEE Transactions on Systems, Man, and Cybernetics:Systems, 2017, 48 (99): 11- 20
[6]   LI X, DING Q, SUN J Q Remaining useful life estimation in prognostics using deep convolution neural networks[J]. Reliability Engineering and System Safety, 2018, 172: 1- 11
[7]   SAYAH M, GUEBLI D, MASRY Z A, et al Robustness testing framework for RUL prediction Deep LSTM networks[J]. ISA Transactions, 2021, 113 (1): 28- 38
[8]   KHELIF R, CHEBEL MORELLO B, MALINOWSKI S, et al Direct remaining useful life estimation based on support vector regression[J]. IEEE Transactions on Industrial Electronics, 2017, 64 (3): 2276- 2285
doi: 10.1109/TIE.2016.2623260
[9]   CANNARILE F, BARALDI P, ZIO E An evidential similarity-based regression method for the prediction of equipment remaining useful life in presence of incomplete degradation trajectories[J]. Fuzzy Sets and Systems, 2019, 367 (15): 36- 50
[10]   ZHOU Z H, WU J, TANG W Ensembling neural networks: many could be better than all[J]. Artificial Intelligence, 2002, 137 (1-2): 239- 263
doi: 10.1016/S0004-3702(02)00190-X
[11]   刘如辉, 黄炜平, 王凯, 等 半监督约束集成的快速密度峰值聚类算法[J]. 浙江大学学报: 工学版, 2018, 52 (11): 2191- 2200
LIU Ru-hui, HUANG Wei-ping, WANG Kai, et al Semi-supervised constraint ensemble clustering by fast search and find of density peaks[J]. Journal of Zhejiang University: Engineering Science, 2018, 52 (11): 2191- 2200
[12]   HERNÁNDEZ-LOBATO DANIEL, MARTÍNEZ MUÑOZ, A SUÁREZ Statistical instance based pruning in ensembles of independent classifiers[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009, 31 (2): 364- 369
[13]   LIU Z, DAI Q, LIU N Ensemble selection by GRASP[J]. Applied Intelligence, 2014, 41 (1): 128- 144
doi: 10.1007/s10489-013-0510-0
[14]   XIA T B, JIANG Y M, ZHUO P C, et al Dual-ensemble multi-feedback neural network for gearbox fault diagnosis[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 1- 10
[15]   ZHANG C, LIM P, QIN A K, et al Multi-objective deep belief networks ensemble for remaining useful life estimation in prognostics[J]. IEEE Transactions on Neural Networks and Learning Systems, 2017, 28 (10): 2306- 2318
doi: 10.1109/TNNLS.2016.2582798
[16]   BROWN G, WYATT J, HARRIS R, et al Diversity creation methods: a survey and categorization[J]. Information Fusion, 2005, 6 (1): 5- 20
doi: 10.1016/j.inffus.2004.04.004
[17]   XIA T B, ZHUO P C, XIAO L, et al Multi-stage fault diagnosis framework for rolling bearing based on OHF Elman AdaBoost-bagging algorithm[J]. Neurocomputing, 2021, 433 (14): 237- 251
[18]   XIA T B, SHU J Q, XU Y H, et al Multiscale similarity ensemble framework for remaining useful life prediction[J]. Measurement, 2022, 188: 110565
doi: 10.1016/j.measurement.2021.110565
[19]   李瑶琦, 周鑫, 高卫益, 等 基于Stacking集成学习的急诊患者到达预测[J]. 工业工程与管理, 2019, 24 (6): 180- 187
LI Yao-qi, ZHOU Xin, GAO Wei-yi, et al Emergency patient arrival forecast based on stacking ensemble learning[J]. Industrial Engineering and Management, 2019, 24 (6): 180- 187
doi: 10.19495/j.cnki.1007-5429.2019.06.022
[20]   XIA T B, SONG Y, ZHENG Y, et al An ensemble framework based on convolutional bi-directional LSTM with multiple time windows for remaining useful life estimation[J]. Computers in Industry, 2020, 115: 103182
doi: 10.1016/j.compind.2019.103182
[21]   许昱晖, 舒俊清, 宋亚, 等 基于多时间尺度相似性的涡扇发动机寿命预测[J]. 浙江大学学报: 工学版, 2021, 55 (10): 1937- 1947
XU Yu-hui, SHU Jun-qing, SONG Ya, et al Remaining useful life prediction of turbofan engine based on similarity in multiple time scales[J]. Journal of Zhejiang University: Engineering Science, 2021, 55 (10): 1937- 1947
[22]   KROGH A, VEDELSBY J Neural network ensembles, cross validation, and active learning[J]. Advances in Neural Information Processing Systems, 1995, 7: 231- 238
[23]   SAXENA A, KAI G, SIMON D, et al. Damage propagation modeling for aircraft engine run-to-failure simulation [C]// International Conference on Prognostics and Health Management. Denver, CO, USA: IEEE, 2008: 1-9.
[24]   HEIMES F O. Recurrent neural networks for remaining useful life estimation [C]// Proceedings of the International Conference on Prognostics and Health Management. Washington, D. C. , USA: IEEE, 2008: 1-6.
[25]   ZHENG S, RISTOVSKI K, FARAHAT A, et al. Long short-term memory network for remaining useful life estimation [C]// IEEE International Conference on Prognostics and Health Management (ICPHM 2014). Dallas, TX, USA: IEEE, 2017: 88-95.
[26]   KONG Z, CUI Y, XIA Z, et al Convolution and long short-term memory hybrid deep neural networks for remaining useful life prognostics[J]. Applied Sciences, 2019, 9 (19): 4156
doi: 10.3390/app9194156
[27]   YU W, KIM I Y, MECHEFSKE C An improved similarity-based prognostic algorithm for RUL estimation using an RNN autoencoder scheme[J]. Reliability Engineering and System Safety, 2020, 199: 106926
[1] Ren-peng MO,Xiao-sheng SI,Tian-mei LI,Xu ZHU. Bearing life prediction based on multi-scale features and attention mechanism[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(7): 1447-1456.
[2] Yu-hui XU,Jun-qing SHU,Ya SONG,Yu ZHENG,Tang-bin XIA. Remaining useful life prediction of turbofan engine based on similarity in multiple time scales[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(10): 1937-1947.
[3] Tian-zhong HU,Jian-bo YU. Life prediction of lithium-ion batteries based on multiscale decomposition and deep learning[J]. Journal of ZheJiang University (Engineering Science), 2019, 53(10): 1852-1864.