注意力卷积GRU自编码器及其在工业过程监控的应用

doi:10.3785/j.issn.1008-973X.2021.09.005

浙江大学学报(工学版)

2021, Vol. 55

Issue (9): 1643-1651 DOI: 10.3785/j.issn.1008-973X.2021.09.005

机械工程、能源工程

注意力卷积GRU自编码器及其在工业过程监控的应用

刘兴(

),余建波*(

)

同济大学机械与能源工程学院，上海 201804

Attention convolutional GRU-based autoencoder and its application in industrial process monitoring

Xing LIU(

),Jian-bo YU*(

)

School of Mechanical Engineering, Tongji University, Shanghai 201804, China

全文: PDF(1133 KB) HTML

摘要：

针对现有故障检测算法难以深入并准确地提取数据内在信息的问题，提出注意力卷积门控循环单元自编码器（CGRUA-AE）深度神经网络和基于CGRUA-AE的过程故障检测方法. 采用卷积门控循环单元（ConvGRU）有效地提取输入数据的空间和时间特征；建立基于ConvGRU的自编码器，采用无监督学习对时间序列数据进行特征提取，引入注意力机制对相应的特征进行加权计算，实现对关键特征的有效选择；分别在特征空间与残差空间上建立基于T ²、SPE统计量的过程监控模型，实现对多元数据有效的特征提取和故障检测. 数值案例和田纳西?伊士曼过程故障检测结果表明，CGRUA-AE具有良好的特征提取能力和故障检测能力，性能优于常用的过程故障检测方法.

关键词： 过程监控; 故障检测; 深度学习; 自编码器; 卷积门控循环单元（ConvGRU）; 注意力

Abstract:

A new deep neural network with attention convolutional gated recurrent unit-based autoencoder (CGRUA-AE) and a process fault detection method based on CGRUA-AE were proposed aiming at the problem that the existing fault detection algorithms were difficult to extract the internal information of data deeply and accurately. First, a convolutional gated recurrent unit (ConvGRU) was effectively extracted the spatial and temporal features of input data. Secondly, an auto-encoder based on ConvGRU was established, using unsupervised learning to extract features from time series data, introducing an attention mechanism to calculate the weight of corresponding features to realize the effective selection of key features. Finally, the process monitoring model based on $ {T}^{2} $ and SPE statistics were established in feature space and residual space respectively to realizes effective feature extraction and fault detection for multivariate data. Numerical case and Tennessee-Eastman process fault detection results show that CGRUA-AE has good feature extraction ability and fault detection ability, and its performance is superior to the common process fault detection methods.

Key words: process monitoring fault detection deep learning autoencoder convolutional gated recurrent unit (ConvGRU) attention

收稿日期: 2020-08-24 出版日期: 2021-10-20

CLC:

TP 277

基金资助: 国家自然科学基金资助项目 (71771173)；上海市科学技术委员会“科技创新行动计划”高新技术领域资助项目（19511106303）

通讯作者: 余建波 E-mail: 953465408@qq.com;jbyu@tongji.edu.cn

作者简介: 刘兴（1996—），男，硕士生，从事过程控制研究. orcid.org/0000-0001-7153-8009. E-mail： 953465408@qq.com

	服务
	把本文推荐给朋友
	加入引用管理器
	E-mail Alert
	作者相关文章
	刘兴
	余建波

引用本文:

刘兴,余建波. 注意力卷积GRU自编码器及其在工业过程监控的应用[J]. 浙江大学学报(工学版), 2021, 55(9): 1643-1651.

Xing LIU,Jian-bo YU. Attention convolutional GRU-based autoencoder and its application in industrial process monitoring. Journal of ZheJiang University (Engineering Science), 2021, 55(9): 1643-1651.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2021.09.005 或 https://www.zjujournals.com/eng/CN/Y2021/V55/I9/1643

图 1 ConvGRU结构

图 2 CGRUA-AE的网络结构

图 3 AM的作用流程

图 4 基于CGRUA-AE的过程监控流程

图 5 故障1的特征可视化

表 1 4种故障检测方法在T2统计量下的FDR/DR

表 2 4种故障检测方法在SPE统计量下的FDR/DR

图 6 4种故障检测方法对TEP故障5的监控结果

表 3 4种检测方法在T2统计量的TEP故障FDR/DR

表 4 4种检测方法在SPE统计量的TEP故障FDR/DR

表 5 CGRUA-AE与深度学习检测方法的TEP故障FDR/DR

图 7 CGRUA-AE与CGRU-AE的损失值

表 6 CGRUA-AE与CGRU-AE的故障检测率

1	YIN S, YANG X, KARIMI H R Data-driven adaptive observer for fault diagnosis[J]. Mathematical Problems in Engineering, 2012, 2012: 832836
2	XIE X, SUN W, CHEUNG K An advanced PLS approach for key performance indicator-related prediction and diagnosis in case of outliers[J]. IEEE Transactions on Industrial Electronics, 2016, 63 (4): 2587- 2594
3	YANG Q, GE S S, SUN Y Adaptive actuator fault tolerant control for uncertain nonlinear systems with multiple actuators[J]. Automatica, 2015, 60: 92- 99 doi: 10.1016/j.automatica.2015.07.006
4	WAN Z, LI J, GAO Y Monitoring and diagnosis process of abnormal consumption on smart power grid[J]. Neural Computing and Applications, 2018, 30 (1): 21- 28 doi: 10.1007/s00521-016-2719-4
5	GE Z Q Review on data-driven modeling and monitoring for plant wide industrial processes[J]. Chemometrics and Intelligent Laboratory Systems, 2017, 171: 16- 25 doi: 10.1016/j.chemolab.2017.09.021
6	NOR N M, HASSAN C R C, HUSSAIN M A A review of data-driven fault detection and diagnosis methods: applications in chemical process systems[J]. Reviews in Chemical Engineering, 2019, 36 (4): 513- 553
7	JIANG Q, YAN X, HUANG B Performance-driven distributed PCA process monitoring based on fault-relevant variable selection and bayesian inference[J]. IEEE Transactions on Industrial Electronics, 2015, 63 (1): 377- 386
8	DENG X, TIAN X, CHEN S, et al Nonlinear process fault diagnosis based on serial principal component analysis[J]. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29 (3): 560- 572 doi: 10.1109/TNNLS.2016.2635111
9	ZHONG B, WANG J, ZHOU J, et al Quality-related statistical process monitoring method based on global and local partial least-squares projection[J]. Industrial and Engineering Chemistry Research, 2016, 55 (6): 1609- 1622 doi: 10.1021/acs.iecr.5b02559
10	YU G Fault feature extraction using independent component analysis with reference and its application on fault diagnosis of rotating machinery[J]. Neural Computing and Applications, 2015, 26 (1): 187- 198 doi: 10.1007/s00521-014-1726-6
11	ZHONG S, WEN Q, GE Z Semi-supervised Fisher discriminant analysis model for fault classification in industrial processes[J]. Chemometrics and Intelligent Laboratory Systems, 2014, 138: 203- 211 doi: 10.1016/j.chemolab.2014.08.008
12	LEE J M, YOO C K, CHOI S W, et al Nonlinear process monitoring using kernel principal component analysis[J]. Chemical Engineering Science, 2004, 59 (1): 223- 234 doi: 10.1016/j.ces.2003.09.012
13	JIANG Q, YAN X Parallel PCA–KPCA for nonlinear process monitoring[J]. Control Engineering Practice, 2018, 80 (9): 17- 25
14	DEWI Y N, RIANA D, MANTORO T. Improving nave bayes performance in single image pap smear using weighted principal component analysis (WPCA) [C]// 2017 International Conference on Computing, Engineering, and Design (ICCED). Jakarta: IEEE, 2018: 1-5
15	WANG K, JUNG H C, ZHI S Performance analysis of dynamic PCA for closed-loop process monitoring and its improvement by output oversampling scheme[J]. IEEE Transactions on Control Systems and Technology, 2019, 27 (1): 378- 85 doi: 10.1109/TCST.2017.2765621
16	HEO S, LEE J H Fault detection and classification using artificial neural networks[J]. IFAC-PapersOnLine, 2018, 51 (18): 470- 475 doi: 10.1016/j.ifacol.2018.09.380
17	YANG C, HOU J Fed-batch fermentation penicillin process fault diagnosis and detection based on support vector machine[J]. Neurocomputing, 2016, 190 (19): 117- 123
18	LI Y, LIU Y, ZHANG C Discriminant diffusion maps based K-nearest-neighbour for batch process fault detection [J]. Canadian Journal of Chemical Engineering, 2018, 96 (2): 484- 496 doi: 10.1002/cjce.23003
19	KIM Y, KIM S B Optimal false alarm-controlled support vector data description for multivariate process monitoring[J]. Journal of Process Control, 2017, 65: 1- 14
20	黄健, 杨旭基于在线加权慢特征分析的故障检测算法[J]. 上海交通大学学报, 2020, 54 (11): 1142- 1150 HUANG Jian, YANG Xu Online Weighted Based Slow Feature Analysis Fault Detection Algorithm[J]. Journal of Shanghai Jiao Tong University, 2020, 54 (11): 1142- 1150
21	HINTON, G Reducing the dimensionality of data with neural networks[J]. Science, 2006, 313 (5786): 504- 507 doi: 10.1126/science.1127647
22	LECUN Y, BENGIO Y, HINTON G Deep learning[J]. Nature, 2015, 521 (7553): 436- 444 doi: 10.1038/nature14539
23	LIU Y, FAN Y, CHEN J Flame images for oxygen content prediction of combustion systems using DBN[J]. Energy Fuel, 2017, 31 (8): 8776- 8783 doi: 10.1021/acs.energyfuels.7b00576
24	XUAN Q, FANG BW, LIU Y, et al Automatic pearl classification machine based on a multistream convolutional neural network[J]. IEEE Transactions on Industrial Electronics, 2018, 65 (8): 6538- 6547 doi: 10.1109/TIE.2017.2784394
25	KRIZHEVSKY A, SUTSKEVER I, HINTON G ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60 (6): 84- 90 doi: 10.1145/3065386
26	ZHANG X Y, YIN F, ZHANG Y M, et al Drawing and recognizing chinese characters with recurrent neural network[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40 (4): 849- 862 doi: 10.1109/TPAMI.2017.2695539
27	LAULY S, LAROCHELLE H, KHAPRA M, et al An autoencoder approach to learning bilingual word representations[J]. Advances in Neural Information Processing Systems, 2014, 3: 1853- 1861
28	ZHANG Z P, ZHAO J S A deep belief network based fault diagnosis model for complex chemical processes[J]. Computers and Chemical Engineering, 2017, 107: 395- 407 doi: 10.1016/j.compchemeng.2017.02.041
29	TANG P, PENG K, ZHANG K A deep belief network-based fault detection method for nonlinear processes[J]. IFAC-PapersOnLine, 2018, 51 (24): 9- 14 doi: 10.1016/j.ifacol.2018.09.522
30	李元, 冯成成基于一维卷积神经网络深度学习的工业过程故障检测[J]. 测控技术, 2019, 38 (9): 36- 40 LI Yuan, FENG Cheng-cheng Fault detection of industrial process based on deep learning of one-dimensional convolution neural network[J]. Measurement and Control Technology, 2019, 38 (9): 36- 40
31	WU H, ZHAO J S Deep convolutional neural network model based chemical process fault diagnosis[J]. Computers and Chemical Engineering, 2018, 115: 185- 197 doi: 10.1016/j.compchemeng.2018.04.009
32	ZHANG Z H, TENG J, LI S H, et al Automated feature learning for nonlinear process monitoring: an approach using stacked denoising autoencoder and k-nearest neighbor rule[J]. Journal of Process Control, 2018, 64: 49- 61 doi: 10.1016/j.jprocont.2018.02.004
33	CHO K, MERRIENBOER B V, GULCEHRE C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation [C]// Proceeding of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Doha: [s.n.], 2014: 1724-1734.
34	HOCHREITER S, SCHMIDHUBER J Long short-term memory[J]. Neural Computation, 1997, 9 (8): 1735- 1780 doi: 10.1162/neco.1997.9.8.1735
35	SHI X, GAO Z, LAUSEN L. Deep learning for precipitation nowcasting: a benchmark and a new model [C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach: [s.n.], 2017: 5617-5627.
36	MNIH V, HEESS N, GRAVES A. Recurrent models of visual attention [EB/OL]. [2020-07-24]. http://arxiv. org/pdf/1406.6247.pdf.
37	CHEN Q, WYNNE R J, GOULDING P, et al The application of principal component analysis and kernel density estimation to enhance process monitoring[J]. Control Engineering Practice, 2000, 8 (5): 531- 543 doi: 10.1016/S0967-0661(99)00191-4
38	LAURENS V D M, HINTON G Visualizing data using t-SNE[J]. Journal of Machine Learning Research, 2008, 9 (86): 2579- 2605

[1]	鞠晓臣,赵欣欣,钱胜胜. 基于自注意力机制的桥梁螺栓检测算法[J]. 浙江大学学报(工学版), 2022, 56(5): 901-908.
[2]	白文超,韩希先,王金宝. 基于条件生成模型的高效近似查询处理框架[J]. 浙江大学学报(工学版), 2022, 56(5): 995-1005.
[3]	何立,庞善民. 结合年龄监督和人脸先验的语音-人脸图像重建[J]. 浙江大学学报(工学版), 2022, 56(5): 1006-1016.
[4]	陈成,张皞,李永强,冯远静. 关系生成图注意力网络的知识图谱链接预测[J]. 浙江大学学报(工学版), 2022, 56(5): 1025-1034.
[5]	王友卫,童爽,凤丽洲,朱建明,李洋,陈福. 基于图卷积网络的归纳式微博谣言检测新方法[J]. 浙江大学学报(工学版), 2022, 56(5): 956-966.
[6]	张雪芹,李天任. 基于Cycle-GAN和改进DPN网络的乳腺癌病理图像分类[J]. 浙江大学学报(工学版), 2022, 56(4): 727-735.
[7]	许萌,王丹,李致远,陈远方. IncepA-EEGNet: 融合Inception网络和注意力机制的P300信号检测方法[J]. 浙江大学学报(工学版), 2022, 56(4): 745-753, 782.
[8]	柳长源,何先平,毕晓君. 融合注意力机制的高效率网络车型识别[J]. 浙江大学学报(工学版), 2022, 56(4): 775-782.
[9]	陈巧红,裴皓磊,孙麒. 基于视觉关系推理与上下文门控机制的图像描述[J]. 浙江大学学报(工学版), 2022, 56(3): 542-549.
[10]	褚晶辉,史李栋,井佩光,吕卫. 适用于目标检测的上下文感知知识蒸馏网络[J]. 浙江大学学报(工学版), 2022, 56(3): 503-509.
[11]	程若然,赵晓丽,周浩军,叶翰辰. 基于深度学习的中文字体风格转换研究综述[J]. 浙江大学学报(工学版), 2022, 56(3): 510-519, 530.
[12]	农元君,王俊杰,陈红,孙文涵,耿慧,李书悦. 基于注意力机制和编码-解码架构的施工场景图像描述方法[J]. 浙江大学学报(工学版), 2022, 56(2): 236-244.
[13]	刘英莉,吴瑞刚,么长慧,沈韬. 铝硅合金实体关系抽取数据集的构建方法[J]. 浙江大学学报(工学版), 2022, 56(2): 245-253.
[14]	董红召,方浩杰,张楠. 旋转框定位的多尺度再生物品目标检测算法[J]. 浙江大学学报(工学版), 2022, 56(1): 16-25.
[15]	王鑫,陈巧红,孙麒,贾宇波. 基于关系推理与门控机制的视觉问答方法[J]. 浙江大学学报(工学版), 2022, 56(1): 36-46.

Viewed

Full text

Abstract

Cited

Shared

Discussed