Please use this identifier to cite or link to this item: https://hdl.handle.net/10356/89770
Title: Numerical analysis near singularities in RBF networks
Authors: Guo, Weili
Ong, Yew-Soon
Hervas, Jaime Rubio
Zhao, Junsheng
Zhang, Kanjian
Wang, Hai
Wei, Haikun
Keywords: DRNTU::Engineering::Computer science and engineering
RBF Networks
Singularity
Issue Date: 2018
Source: Guo, W., Wei, H., Ong, Y-S., Hervas, J. R., Zhao, J., Wang, H., & Zhang, K. (2018). Numerical analysis near singularities in RBF networks. Journal of Machine Learning Research, 19, 1-39.
Series/Report no.: Journal of Machine Learning Research
Abstract: The existence of singularities often affects the learning dynamics in feedforward neural networks. In this paper, based on theoretical analysis results, we numerically analyze the learning dynamics of radial basis function (RBF) networks near singularities to understand to what extent singularities influence the learning dynamics. First, we show the explicit expression of the Fisher information matrix for RBF networks. Second, we demonstrate through numerical simulations that the singularities have a significant impact on the learning dynamics of RBF networks. Our results show that overlap singularities mainly have influence on the low dimensional RBF networks and elimination singularities have a more significant impact to the learning processes than overlap singularities in both low and high dimensional RBF networks, whereas the plateau phenomena are mainly caused by the elimination singularities. The results can also be the foundation to investigate the singular learning dynamics in deep feedforward neural networks.
URI: https://hdl.handle.net/10356/89770
http://hdl.handle.net/10220/46400
ISSN: 1532-4435
Rights: © 2018 Weili Guo, Haikun Wei, Yew-Soon Ong, Jaime Rubio Hervas, Junsheng Zhao, Hai Wang and Kanjian Zhang. License: CC-BY 4.0, see https://creativecommons.org/licenses/by/4.0/. Attribution requirements are provided at http://jmlr.org/papers/v19/16-210.html.
Fulltext Permission: open
Fulltext Availability: With Fulltext
Appears in Collections:SCSE Journal Articles

Files in This Item:
File Description SizeFormat 
Numerical Analysis near Singularities in RBF Networks.pdf2.03 MBAdobe PDFThumbnail
View/Open

Google ScholarTM

Check

Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.