dc.contributor.authorXu, Zhao
dc.contributor.authorSong, Qing
dc.contributor.authorWang, Danwei
dc.contributor.authorFan, Haijin
dc.identifier.citationXu, Z., Song, Q., Wang, D., & Fan, H. (2012). A training algorithm and stability analysis for recurrent neural networks. 2012 15th International Conference on Information Fusion (FUSION), 2285-2292.en_US
dc.description.abstractTraining of recurrent neural networks (RNNs) introduces considerable computational complexities due to the need for gradient evaluations. How to get fast convergence speed and low computational complexity remains a challenging and open topic. Besides, the transient response of learning process of RNNs is a critical issue, especially for on-line applications. Conventional RNNs training algorithms such as the backpropagation through time (BPTT) and real-time recurrent learning (RTRL) have not adequately satisfied these requirements because they often suffer from slow convergence speed. If a large learning rate is chosen to improve performance, the training process may become unstable in terms of weight divergence. In this paper, a novel training algorithm of RNN, named robust recurrent simultaneous perturbation stochastic approximation (RRSPSA), is developed with a specially designed recurrent hybrid adaptive parameter and adaptive learning rates. RRSPSA is a powerful novel twin-engine simultaneous perturbation stochastic approximation (SPSA) type of RNN training algorithm. It utilizes specific designed three adaptive parameters to maximize training speed for recurrent training signal while exhibiting certain weight convergence properties with only two objective function measurements as the original SPSA algorithm. The RRSPSA is proved with guaranteed weight convergence and system stability in the sense of Lyapunov function. Computer simulations were carried out to demonstrate applicability of the theoretical results.en_US
dc.rights© 2012 International Society of Information Fusion. This paper was published in 2012 15th International Conference on Information Fusion (FUSION) and is made available as an electronic reprint (preprint) with permission of International Society of Information Fusion. The paper can be found at the following official URL: http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=6290583&url=http%3A%2F%2Fieeexplore.ieee.org%2Fiel5%2F6269381%2F6289713%2F06290583.pdf%3Farnumber%3D6290583 .  One print or electronic copy may be made for personal use only. Systematic or multiple reproduction, distribution to multiple locations via electronic or other means, duplication of any material in this paper for a fee or for commercial purposes, or modification of the content of the paper is prohibited and is subject to penalties under law.en_US
dc.subjectDRNTU::Engineering::Electrical and electronic engineering
dc.titleA training algorithm and stability analysis for recurrent neural networksen_US
dc.typeConference Paper
dc.contributor.conferenceInternational Conference on Information Fusion (FUSION) (15th : 2012)en_US
dc.contributor.schoolSchool of Electrical and Electronic Engineeringen_US
dc.description.versionPublished versionen_US

Files in this item


This item appears in the following Collection(s)

Show simple item record