Please wait a minute...
Journal of ZheJiang University (Engineering Science)  2020, Vol. 54 Issue (12): 2414-2422    DOI: 10.3785/j.issn.1008-973X.2020.12.016
    
Application load forecasting method based on multi-layer bidirectional LSTM and improved PSO algorithm
Liang CAI(),Hong-cen ZHOU,Heng BAI,Zhen-gong CAI*(),Ke-ting YIN,Yi-jun BEI
College of Software Technology, Zhejiang University, Ningbo 315000, China
Download: HTML     PDF(1180KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

In order to solve the problems of low accuracy of commonly used time series prediction algorithms and difficulty in tuning parameters, a load prediction method was proposed based on a multilayer bidirectional long short-term memory (BiLSTM) neural network. The method includes network model design, adaptive parameter setting, and improved particle swarm optimization (PSO). The data was input into the BiLSTM network model for training and uses an adaptive algorithm for automatically parameter adjustment. The method uses a model evaluation method based on the benchmark model to calculate the fitness of the improved particle swarm optimization algorithm. The method uses improved particle swarm optimization to optimize the prediction results of the model. The average absolute percentage error of this method was reduced by 3.6% to 7.2% compared with several typical forecasting algorithms, and the training time was reduced by more than 10%, Through experimental comparison with a variety of typical time series prediction algorithms. Experimental results show that the method has higher accuracy and stronger applicability in time series forecasting, and provide an important scientific basis for the use of load forecast results for elastic expansion and contraction.



Key wordsload forecaste      bidirectional long-short-term memory (BiLSTM)      particle swarm optimization (PSO)      adaptive algorithm      multi index fusion     
Received: 31 December 2019      Published: 31 December 2020
CLC:  TU 111  
Corresponding Authors: Zhen-gong CAI     E-mail: leoncai@zju.edu.cn;cstcaizg@zju.edu.cn
Cite this article:

Liang CAI,Hong-cen ZHOU,Heng BAI,Zhen-gong CAI,Ke-ting YIN,Yi-jun BEI. Application load forecasting method based on multi-layer bidirectional LSTM and improved PSO algorithm. Journal of ZheJiang University (Engineering Science), 2020, 54(12): 2414-2422.

URL:

http://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2020.12.016     OR     http://www.zjujournals.com/eng/Y2020/V54/I12/2414


基于多层BiLSTM和改进粒子群算法的应用负载预测方法

为了解决常用时序预测算法精度不高和调参困难的问题,提出基于多层双向长短期记忆(BiLSTM)神经网络的负载预测方法,包括网络模型设计、自适应参数设置和改进粒子群算法优化等步骤. 将数据输入网络模型中进行训练,使用自适应算法进行自动调参;采用基于基准模型的多指标融合的模型评价方法,计算改进粒子群算法的适应度;使用改进粒子群算法优化模型的预测结果. 通过与多种典型时间序列预测算法的实验对比,方法的预测平均绝对百分比误差减小3.6%~7.2%,训练时间缩短10%以上,实验结果验证了方法在时间序列预测中具有更高的准确性和很强的适用性,为使用负载预测结果进行弹性扩缩容提供了重要的科学依据.


关键词: 负载预测,  双向长短记忆(BiLSTM),  粒子群算法(PSO),  自适应算法,  多指标融合 
Fig.1 LSTM memory cell structure
Fig.2 BiLSTM network structure
Fig.3 Multi-layer BiLSTM network structure diagram
Fig.4 Flowchart of adaptive parameter adjustment algorithm
Fig.5 Application load of request volume in one-day
Fig.6 Application load of request volume in one-week
Fig.7 Application load of request volume in 35 days
模型 RMSE MAE MAPE/% Q
Pa-BiLSTM 121.05 78.67 3.97 0.37
PSO-BiLSTM 157.13 92.53 4.63 0.45
BiLSTM 176.25 105.24 5.65 0.49
Pa-LSTM 158.39 102.84 5.38 0.46
LSTM 267.40 155.03 7.64 0.65
ARIMA(1,1,1) 366.84 167.46 5.87 0.73
HoltWinters 196.13 162.49 8.21 0.58
Prophet 234.92 198.94 11.18 0.67
基准模型 452.22 347.03 29.62 1.00
Tab.1 Load prediction errors comparison
Fig.8 Forecast accuracy of different models under different data sizes
Fig.9 Training time of different models on different data sizes
Fig.10 Comparison chart of fitness between oPaBiLSTM and Pa-BiLSTM
[1]   YAN Y Q, GUO P Predicting resource consumption in a Web server using ARIMA model[J]. Journal of Beijing Institute of Technology, 2014, 23 (4): 502- 510
[2]   CALHEIROS R N, MASOUMI E, RANJAN R, et al Workload prediction using ARIMA model and its impact on cloud applications' QoS[J]. IEEE Transactions on Cloud Computing, 2014, 3 (4): 449- 458
[3]   PRASSANNA J, VENKATARAMAN N Adaptive regressive holt–winters workload prediction and firefly optimized lottery scheduling for load balancing in cloud[J]. Wireless Networks, 2019, 1- 19
[4]   YAZDANIAN P, SHARIFIAN S. Cloud workload prediction using ConvNet and stacked LSTM [C]// 2018 4th Iranian Conference on Signal Processing and Intelligent Systems. Tehran: IEEE, 2018: 83-87.
[5]   谢晓兰, 张征征, 郑强清, 等 基于APMSSGA-LSTM 的容器云资源预测[J]. 大数据, 2019, 5 (6): 62- 72
XIE Xiao-lan, ZHANG Zheng-zheng, ZHENG Qiang-qing, et al Container cloud resource prediction based on APMSSGA-LSTM[J]. Big Data Research, 2019, 5 (6): 62- 72
[6]   LIU J, TAN X Y, WANG Y. CSSAP: software aging prediction for cloud services based on ARIMA-LSTM hybrid model [C]// 2019 IEEE International Conference on Web Services. Milan: IEEE, 2019: 283-290.
[7]   KUMAR S D, SUBHA D P. Prediction of depression from EEG signal using long short term memory [C]// 2019 3rd International Conference on Trends in Electronics and Informatics. Tirunelveli: IEEE, 2019: 1248-1253.
[8]   ZI?BA M, TOMCZAK S K, TOMCZAK J M Ensemble boosted trees with synthetic features generation in application to bankruptcy prediction[J]. Expert Systems with Applications, 2016, 58: 93- 101
doi: 10.1016/j.eswa.2016.04.001
[9]   AKYUZ A O, UYSAL M, BULBUL B A, et al. Ensemble approach for time series analysis in demand forecasting: Ensemble learning [C]// 2017 IEEE International Conference on Innovations in Intelligent Systems and Applications. Gdynia: IEEE, 2017: 7-12.
[10]   BIN Y, YANG Y, SHEN F, et al. Bidirectional long-short term memory for video description [C]// Proceedings of the 24th ACM International Conference on Multimedia. New York: ACM, 2016: 436-440.
[11]   ZENNAKI O, SEMMAR N, BESACIER L. Inducing multilingual text analysis tools using bidirectional recurrent neural networks [C]// 26th International Conference on Computational Linguistics: Technical Papers. Osaka: COLING. 2016: 450-460.
[12]   SCHUSTER M, PALIWAL K K Bidirectional recurrent neural networks[J]. IEEE Transactions on Signal Processing, 1997, 45 (11): 2673- 2681
doi: 10.1109/78.650093
[13]   SUN W, XU L, HUANG X, et al. Bidirectional LSTM for ionospheric vertical total electron content forecasting [C]// 2017 IEEE Visual Communications and Image Processing. St. Petersburg: IEEE, 2017: 1-4.
[14]   HU P, TONG J, WANG J, et al. A hybrid model based on CNN and Bi-LSTM for urban water demand prediction [C]// 2019 IEEE Congress on Evolutionary Computation. Wellington: IEEE, 2019: 1088-1094.
[15]   HAN S, ZHANG F, XI J, et al. Short-term vehicle speed prediction based on convolutional bidirectional LSTM networks [C]// 2019 IEEE Intelligent Transportation Systems Conference. Auckland: IEEE, 2019: 4055-4060.
[16]   KENNEDY J, EBERHART R. Particle swarm optimization [C]// Proceedings of ICNN'95-International Conference on Neural Networks. Perth: IEEE, 1995, 4: 1942-1948.
[17]   CHEN C, TWYCROSS J, GARIBALDI J M A new accuracy measure based on bounded relative error for time series forecasting[J]. PloS One, 2017, 12 (3): e0174202
doi: 10.1371/journal.pone.0174202
[18]   HOCHREITER S, SCHMIDHUBER J Long short-term memory[J]. Neural Computation, 1997, 9 (8): 1735- 1780
doi: 10.1162/neco.1997.9.8.1735
[19]   张利彪, 周春光, 马铭, 等 基于粒子群算法求解多目标优化问题[J]. 计算机研究与发展, 2004, 41 (7): 1286- 1291
ZHANG Li-biao, ZHOU Chun-guang, MA Ming, et al Solving multi-objective optimization problems based on particle swarm optimization[J]. Journal of Computer Research and Development, 2004, 41 (7): 1286- 1291
[20]   DING W, FANG W. Target tracking by sequential random draft particle swarm optimization algorithm [C]// 2018 IEEE International Smart Cities Conference. Kansas City: IEEE, 2018: 1-7.
[21]   何明慧, 徐怡, 王冉, 等 改进的粒子群算法优化神经网络及应用[J]. 计算机工程与应用, 2018, (19): 17
HE Ming-hui, XU Yi, WANG Ran, et al Improved particle swarm optimization neural network and its application[J]. Computer Engineering and Applications, 2018, (19): 17
[1] ZHANG Qing-ke, MENG Xiang-xu, ZHANG Hua-xiang, YANG Bo, LIU Wei-guo. Particle swarm optimization based on random vector partition and learning[J]. Journal of ZheJiang University (Engineering Science), 2018, 52(2): 367-378.
[2] ZHANG De-sheng, LIU An, CHEN Jian, ZHAO Rui-jie, SHI Wei-dong. Multi-objective optimization of horizontal axis tidal current turbine using particle swarm optimization[J]. Journal of ZheJiang University (Engineering Science), 2018, 52(12): 2349-2355.
[3] ZHAO Xiao-dong, LIU Zuo-jun, CHEN Ling-ling, YANG Peng. Approach of running gait recognition for lower limb amputees[J]. Journal of ZheJiang University (Engineering Science), 2018, 52(10): 1980-1988.
[4] CHEN Te-huan, XU Wei-hua, XU Chao, XIE Lei.
Inverse problem solution of pipeline leakage model based on particle swarm optimization and sensitivity analysis
[J]. Journal of ZheJiang University (Engineering Science), 2014, 48(10): 1850-1855.
[5] WEN Hao-xiang, CHEN Long-dao, CAI Zhong-fa. Decorrelation affine projection algorithm and its application[J]. Journal of ZheJiang University (Engineering Science), 2014, 48(1): 136-140.
[6] LIU Fang, SUN Yun, YANG Geng, LIN Hai. Visualization of social network based on particle swarm optimization[J]. Journal of ZheJiang University (Engineering Science), 2013, 47(1): 37-43.
[7] JIE Li-Jun, WANG Pan-Ni, ZHANG Shuai. Modified PSO method for automating transfer function designing
in volume rendering
[J]. Journal of ZheJiang University (Engineering Science), 2010, 44(8): 1466-1472.