Please wait a minute...
Front. Inform. Technol. Electron. Eng.  2012, Vol. 13 Issue (9): 689-701    DOI: 10.1631/jzus.C1200069
    
Modeling deterministic echo state network with loop reservoir
Xiao-chuan Sun, Hong-yan Cui, Ren-ping Liu, Jian-ya Chen, Yun-jie Liu
Beijing Key Laboratory of Network System Architecture and Convergence, Beijing University of Posts and Telecommunications, Beijing 100876, China; ICT Centre, Commonwealth Scientific and Industrial Research Organization, Sydney 1710, Australia
Download:   PDF(0KB)
Export: BibTeX | EndNote (RIS)      

Abstract  Echo state network (ESN), which efficiently models nonlinear dynamic systems, has been proposed as a special form of recurrent neural network. However, most of the proposed ESNs consist of complex reservoir structures, leading to excessive computational cost. Recently, minimum complexity ESNs were proposed and proved to exhibit high performance and low computational cost. In this paper, we propose a simple deterministic ESN with a loop reservoir, i.e., an ESN with an adjacent-feedback loop reservoir. The novel reservoir is constructed by introducing regular adjacent feedback based on the simplest loop reservoir. Only a single free parameter is tuned, which considerably simplifies the ESN construction. The combination of a simplified reservoir and fewer free parameters provides superior prediction performance. In the benchmark datasets and real-world tasks, our scheme obtains higher prediction accuracy with relatively low complexity, compared to the classic ESN and the minimum complexity ESN. Furthermore, we prove that all the linear ESNs with the simplest loop reservoir possess the same memory capacity, arbitrarily converging to the optimal value.

Key wordsEcho state networks      Loop reservoir structure      Memory capacity     
Received: 15 March 2012      Published: 05 September 2012
CLC:  TP183  
Cite this article:

Xiao-chuan Sun, Hong-yan Cui, Ren-ping Liu, Jian-ya Chen, Yun-jie Liu. Modeling deterministic echo state network with loop reservoir. Front. Inform. Technol. Electron. Eng., 2012, 13(9): 689-701.

URL:

http://www.zjujournals.com/xueshu/fitee/10.1631/jzus.C1200069     OR     http://www.zjujournals.com/xueshu/fitee/Y2012/V13/I9/689


Modeling deterministic echo state network with loop reservoir

Echo state network (ESN), which efficiently models nonlinear dynamic systems, has been proposed as a special form of recurrent neural network. However, most of the proposed ESNs consist of complex reservoir structures, leading to excessive computational cost. Recently, minimum complexity ESNs were proposed and proved to exhibit high performance and low computational cost. In this paper, we propose a simple deterministic ESN with a loop reservoir, i.e., an ESN with an adjacent-feedback loop reservoir. The novel reservoir is constructed by introducing regular adjacent feedback based on the simplest loop reservoir. Only a single free parameter is tuned, which considerably simplifies the ESN construction. The combination of a simplified reservoir and fewer free parameters provides superior prediction performance. In the benchmark datasets and real-world tasks, our scheme obtains higher prediction accuracy with relatively low complexity, compared to the classic ESN and the minimum complexity ESN. Furthermore, we prove that all the linear ESNs with the simplest loop reservoir possess the same memory capacity, arbitrarily converging to the optimal value.

关键词: Echo state networks,  Loop reservoir structure,  Memory capacity 
[1] Muhammad Asif Zahoor Raja, Iftikhar Ahmad, Imtiaz Khan, Muhammed Ibrahem Syam, Abdul Majid Wazwaz. Neuro-heuristic computational intelligence for solving nonlinear pantograph systems[J]. Front. Inform. Technol. Electron. Eng., 2017, 18(4): 464-484.
[2] Yu-qing Chen, Yu-pu Diao, Jing-gang Duan, Li-yuan Cui, Jia-yi Zhang. Time-dependent changes in eye-specific segregation in the dorsal lateral geniculate nucleus and superior colliculus of postnatal mice[J]. Front. Inform. Technol. Electron. Eng., 2014, 15(10): 807-812.
[3] Ali Uysal, Raif Bayir. Real-time condition monitoring and fault diagnosis in switched reluctance motors with Kohonen neural network[J]. Front. Inform. Technol. Electron. Eng., 2013, 14(12): 941-952.
[4] Hasan Abbasi Nozari, Hamed Dehghan Banadaki, Mohammad Mokhtare, Somayeh Hekmati Vahed. Intelligent non-linear modelling of an industrial winding process using recurrent local linear neuro-fuzzy networks[J]. Front. Inform. Technol. Electron. Eng., 2012, 13(6): 403-412.
[5] Xin-zheng Xu, Shi-fei Ding, Zhong-zhi Shi, Hong Zhu. Optimizing radial basis function neural network based on rough sets and affinity propagation clustering algorithm[J]. Front. Inform. Technol. Electron. Eng., 2012, 13(2): 131-138.
[6] Dimitrios Theodoridis, Yiannis Boutalis, Manolis Christodoulou. Direct adaptive regulation of unknown nonlinear systems with analysis of the model order problem[J]. Front. Inform. Technol. Electron. Eng., 2011, 12(1): 1-16.
[7] Jian Bao, Yu Chen, Jin-shou Yu. A regeneratable dynamic differential evolution algorithm for neural networks with integer weights[J]. Front. Inform. Technol. Electron. Eng., 2010, 11(12): 939-947.
[8] Yan Deng, Xiang-ning He, Jing Zhao, Yan Xiong, Yan-qun Shen, Jian Jiang. Application of artificial neural network for switching loss modeling in power IGBTs[J]. Front. Inform. Technol. Electron. Eng., 2010, 11(6): 435-443.