|
|
|
| Attention convolutional GRU-based autoencoder and its application in industrial process monitoring |
Xing LIU( ),Jian-bo YU*() |
| School of Mechanical Engineering, Tongji University, Shanghai 201804, China |
|
|
|
Abstract A new deep neural network with attention convolutional gated recurrent unit-based autoencoder (CGRUA-AE) and a process fault detection method based on CGRUA-AE were proposed aiming at the problem that the existing fault detection algorithms were difficult to extract the internal information of data deeply and accurately. First, a convolutional gated recurrent unit (ConvGRU) was effectively extracted the spatial and temporal features of input data. Secondly, an auto-encoder based on ConvGRU was established, using unsupervised learning to extract features from time series data, introducing an attention mechanism to calculate the weight of corresponding features to realize the effective selection of key features. Finally, the process monitoring model based on $ {T}^{2} $ and SPE statistics were established in feature space and residual space respectively to realizes effective feature extraction and fault detection for multivariate data. Numerical case and Tennessee-Eastman process fault detection results show that CGRUA-AE has good feature extraction ability and fault detection ability, and its performance is superior to the common process fault detection methods.
|
|
Received: 24 August 2020
Published: 20 October 2021
|
|
|
| Fund: 国家自然科学基金资助项目 (71771173)；上海市科学技术委员会“科技创新行动计划”高新技术领域资助项目（19511106303） |
|
Corresponding Authors:
Jian-bo YU
E-mail: 953465408@qq.com;jbyu@tongji.edu.cn
|
注意力卷积GRU自编码器及其在工业过程监控的应用
针对现有故障检测算法难以深入并准确地提取数据内在信息的问题，提出注意力卷积门控循环单元自编码器（CGRUA-AE）深度神经网络和基于CGRUA-AE的过程故障检测方法. 采用卷积门控循环单元（ConvGRU）有效地提取输入数据的空间和时间特征；建立基于ConvGRU的自编码器，采用无监督学习对时间序列数据进行特征提取，引入注意力机制对相应的特征进行加权计算，实现对关键特征的有效选择；分别在特征空间与残差空间上建立基于T 2、SPE统计量的过程监控模型，实现对多元数据有效的特征提取和故障检测. 数值案例和田纳西?伊士曼过程故障检测结果表明，CGRUA-AE具有良好的特征提取能力和故障检测能力，性能优于常用的过程故障检测方法.
关键词：
过程监控,
故障检测,
深度学习,
自编码器,
卷积门控循环单元（ConvGRU）,
注意力
|
|
| [1] |
YIN S, YANG X, KARIMI H R Data-driven adaptive observer for fault diagnosis[J]. Mathematical Problems in Engineering, 2012, 2012: 832836
|
|
|
| [2] |
XIE X, SUN W, CHEUNG K An advanced PLS approach for key performance indicator-related prediction and diagnosis in case of outliers[J]. IEEE Transactions on Industrial Electronics, 2016, 63 (4): 2587- 2594
|
|
|
| [3] |
YANG Q, GE S S, SUN Y Adaptive actuator fault tolerant control for uncertain nonlinear systems with multiple actuators[J]. Automatica, 2015, 60: 92- 99
doi: 10.1016/j.automatica.2015.07.006
|
|
|
| [4] |
WAN Z, LI J, GAO Y Monitoring and diagnosis process of abnormal consumption on smart power grid[J]. Neural Computing and Applications, 2018, 30 (1): 21- 28
doi: 10.1007/s00521-016-2719-4
|
|
|
| [5] |
GE Z Q Review on data-driven modeling and monitoring for plant wide industrial processes[J]. Chemometrics and Intelligent Laboratory Systems, 2017, 171: 16- 25
doi: 10.1016/j.chemolab.2017.09.021
|
|
|
| [6] |
NOR N M, HASSAN C R C, HUSSAIN M A A review of data-driven fault detection and diagnosis methods: applications in chemical process systems[J]. Reviews in Chemical Engineering, 2019, 36 (4): 513- 553
|
|
|
| [7] |
JIANG Q, YAN X, HUANG B Performance-driven distributed PCA process monitoring based on fault-relevant variable selection and bayesian inference[J]. IEEE Transactions on Industrial Electronics, 2015, 63 (1): 377- 386
|
|
|
| [8] |
DENG X, TIAN X, CHEN S, et al Nonlinear process fault diagnosis based on serial principal component analysis[J]. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29 (3): 560- 572
doi: 10.1109/TNNLS.2016.2635111
|
|
|
| [9] |
ZHONG B, WANG J, ZHOU J, et al Quality-related statistical process monitoring method based on global and local partial least-squares projection[J]. Industrial and Engineering Chemistry Research, 2016, 55 (6): 1609- 1622
doi: 10.1021/acs.iecr.5b02559
|
|
|
| [10] |
YU G Fault feature extraction using independent component analysis with reference and its application on fault diagnosis of rotating machinery[J]. Neural Computing and Applications, 2015, 26 (1): 187- 198
doi: 10.1007/s00521-014-1726-6
|
|
|
| [11] |
ZHONG S, WEN Q, GE Z Semi-supervised Fisher discriminant analysis model for fault classification in industrial processes[J]. Chemometrics and Intelligent Laboratory Systems, 2014, 138: 203- 211
doi: 10.1016/j.chemolab.2014.08.008
|
|
|
| [12] |
LEE J M, YOO C K, CHOI S W, et al Nonlinear process monitoring using kernel principal component analysis[J]. Chemical Engineering Science, 2004, 59 (1): 223- 234
doi: 10.1016/j.ces.2003.09.012
|
|
|
| [13] |
JIANG Q, YAN X Parallel PCA–KPCA for nonlinear process monitoring[J]. Control Engineering Practice, 2018, 80 (9): 17- 25
|
|
|
| [14] |
DEWI Y N, RIANA D, MANTORO T. Improving nave bayes performance in single image pap smear using weighted principal component analysis (WPCA) [C]// 2017 International Conference on Computing, Engineering, and Design (ICCED). Jakarta: IEEE, 2018: 1-5
|
|
|
| [15] |
WANG K, JUNG H C, ZHI S Performance analysis of dynamic PCA for closed-loop process monitoring and its improvement by output oversampling scheme[J]. IEEE Transactions on Control Systems and Technology, 2019, 27 (1): 378- 85
doi: 10.1109/TCST.2017.2765621
|
|
|
| [16] |
HEO S, LEE J H Fault detection and classification using artificial neural networks[J]. IFAC-PapersOnLine, 2018, 51 (18): 470- 475
doi: 10.1016/j.ifacol.2018.09.380
|
|
|
| [17] |
YANG C, HOU J Fed-batch fermentation penicillin process fault diagnosis and detection based on support vector machine[J]. Neurocomputing, 2016, 190 (19): 117- 123
|
|
|
| [18] |
LI Y, LIU Y, ZHANG C Discriminant diffusion maps based K-nearest-neighbour for batch process fault detection [J]. Canadian Journal of Chemical Engineering, 2018, 96 (2): 484- 496
doi: 10.1002/cjce.23003
|
|
|
| [19] |
KIM Y, KIM S B Optimal false alarm-controlled support vector data description for multivariate process monitoring[J]. Journal of Process Control, 2017, 65: 1- 14
|
|
|
| [20] |
黄健, 杨旭 基于在线加权慢特征分析的故障检测算法[J]. 上海交通大学学报, 2020, 54 (11): 1142- 1150
HUANG Jian, YANG Xu Online Weighted Based Slow Feature Analysis Fault Detection Algorithm[J]. Journal of Shanghai Jiao Tong University, 2020, 54 (11): 1142- 1150
|
|
|
| [21] |
HINTON, G Reducing the dimensionality of data with neural networks[J]. Science, 2006, 313 (5786): 504- 507
doi: 10.1126/science.1127647
|
|
|
| [22] |
LECUN Y, BENGIO Y, HINTON G Deep learning[J]. Nature, 2015, 521 (7553): 436- 444
doi: 10.1038/nature14539
|
|
|
| [23] |
LIU Y, FAN Y, CHEN J Flame images for oxygen content prediction of combustion systems using DBN[J]. Energy Fuel, 2017, 31 (8): 8776- 8783
doi: 10.1021/acs.energyfuels.7b00576
|
|
|
| [24] |
XUAN Q, FANG BW, LIU Y, et al Automatic pearl classification machine based on a multistream convolutional neural network[J]. IEEE Transactions on Industrial Electronics, 2018, 65 (8): 6538- 6547
doi: 10.1109/TIE.2017.2784394
|
|
|
| [25] |
KRIZHEVSKY A, SUTSKEVER I, HINTON G ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60 (6): 84- 90
doi: 10.1145/3065386
|
|
|
| [26] |
ZHANG X Y, YIN F, ZHANG Y M, et al Drawing and recognizing chinese characters with recurrent neural network[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40 (4): 849- 862
doi: 10.1109/TPAMI.2017.2695539
|
|
|
| [27] |
LAULY S, LAROCHELLE H, KHAPRA M, et al An autoencoder approach to learning bilingual word representations[J]. Advances in Neural Information Processing Systems, 2014, 3: 1853- 1861
|
|
|
| [28] |
ZHANG Z P, ZHAO J S A deep belief network based fault diagnosis model for complex chemical processes[J]. Computers and Chemical Engineering, 2017, 107: 395- 407
doi: 10.1016/j.compchemeng.2017.02.041
|
|
|
| [29] |
TANG P, PENG K, ZHANG K A deep belief network-based fault detection method for nonlinear processes[J]. IFAC-PapersOnLine, 2018, 51 (24): 9- 14
doi: 10.1016/j.ifacol.2018.09.522
|
|
|
| [30] |
李元, 冯成成 基于一维卷积神经网络深度学习的工业过程故障检测[J]. 测控技术, 2019, 38 (9): 36- 40
LI Yuan, FENG Cheng-cheng Fault detection of industrial process based on deep learning of one-dimensional convolution neural network[J]. Measurement and Control Technology, 2019, 38 (9): 36- 40
|
|
|
| [31] |
WU H, ZHAO J S Deep convolutional neural network model based chemical process fault diagnosis[J]. Computers and Chemical Engineering, 2018, 115: 185- 197
doi: 10.1016/j.compchemeng.2018.04.009
|
|
|
| [32] |
ZHANG Z H, TENG J, LI S H, et al Automated feature learning for nonlinear process monitoring: an approach using stacked denoising autoencoder and k-nearest neighbor rule[J]. Journal of Process Control, 2018, 64: 49- 61
doi: 10.1016/j.jprocont.2018.02.004
|
|
|
| [33] |
CHO K, MERRIENBOER B V, GULCEHRE C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation [C]// Proceeding of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Doha: [s.n.], 2014: 1724-1734.
|
|
|
| [34] |
HOCHREITER S, SCHMIDHUBER J Long short-term memory[J]. Neural Computation, 1997, 9 (8): 1735- 1780
doi: 10.1162/neco.1997.9.8.1735
|
|
|
| [35] |
SHI X, GAO Z, LAUSEN L. Deep learning for precipitation nowcasting: a benchmark and a new model [C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach: [s.n.], 2017: 5617-5627.
|
|
|
| [36] |
MNIH V, HEESS N, GRAVES A. Recurrent models of visual attention [EB/OL]. [2020-07-24]. http://arxiv. org/pdf/1406.6247.pdf.
|
|
|
| [37] |
CHEN Q, WYNNE R J, GOULDING P, et al The application of principal component analysis and kernel density estimation to enhance process monitoring[J]. Control Engineering Practice, 2000, 8 (5): 531- 543
doi: 10.1016/S0967-0661(99)00191-4
|
|
|
| [38] |
LAURENS V D M, HINTON G Visualizing data using t-SNE[J]. Journal of Machine Learning Research, 2008, 9 (86): 2579- 2605
|
|
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
| |
Shared |
|
|
|
|
| |
Discussed |
|
|
|
|
