|
|
|
| Adaptive dynamic programming for linear impulse systems |
| Xiao-hua Wang, Juan-juan Yu, Yao Huang, Hua Wang, Zhong-hua Miao |
| School of Mechatronics Engineering and Automation, Shanghai University, Shanghai 200072, China; Shanghai Key Laboratory of Power Station Automation Technology, Shanghai University, Shanghai 200072, China |
|
|
|
Abstract We investigate the optimization of linear impulse systems with the reinforcement learning based adaptive dynamic programming (ADP) method. For linear impulse systems, the optimal objective function is shown to be a quadric form of the pre-impulse states. The ADP method provides solutions that iteratively converge to the optimal objective function. If an initial guess of the pre-impulse objective function is selected as a quadratic form of the pre-impulse states, the objective function iteratively converges to the optimal one through ADP. Though direct use of the quadratic objective function of the states within the ADP method is theoretically possible, the numerical singularity problem may occur due to the matrix inversion therein when the system dimensionality increases. A neural network based ADP method can circumvent this problem. A neural network with polynomial activation functions is selected to approximate the pre-impulse objective function and trained iteratively using the ADP method to achieve optimal control. After a successful training, optimal impulse control can be derived. Simulations are presented for illustrative purposes.
|
|
Received: 27 May 2013
Published: 07 January 2014
|
|
|
线性脉冲系统的自适应动态规划方法
研究目的：针对线性脉冲系统最优控制，研究了基于自适应动态规划的递归方法。通过神经网络逼近最优目标函数，得出最优控制率。求解过程适用于一般脉冲系统，无需初始稳定控制器，为此类系统的最优控制提供理论依据。
创新要点：目前自适应动态规划方法研究局限于连续和离散系统，对脉冲系统研究较少。 本文研究了线性脉冲系统的最优控制问题，运用自适应动态规划思路，完成了脉冲系统相关理论证明，证实了方法的收敛性。通过神经网络逼近最优目标函数，当迭代稳定后，神经网络获得稳定参数，同时获得最优脉冲控制率。
方法提亮：线性脉冲系统的最优目标函数是一个状态二次型，但其中的P矩阵表现为跳跃的脉冲形式。基于此发现，以迭代学习为基础的自适应动态规划方法适用于最优脉冲求解。本文提出的方法避免了直接迭代的矩阵求逆，大大降低了运算量。
重要结论：线性脉冲系统的最优目标函数表现为状态二次型，可通过自适应动态规划方法迭代求解，求解过程稳定。通过神经网络逼近最优目标函数，可避免矩阵求逆，大大降低计算量。
关键词：
脉冲系统,
自适应动态规划,
最优控制,
神经网络
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
| |
Shared |
|
|
|
|
| |
Discussed |
|
|
|
|
