Random vector functional link neural network based deep learning for regression

Abstract

The deep RVFLs are inspired by the principles of the Random Vector Functional Link (RVFL) neural network. Like RVFL, the weights of the hidden layers of the deep RVFLs (dRVFL and edRVFL) are stochastically generated within a suitable range and kept constant throughout the training. In this paper, we test and evaluate the performances of the recently proposed deep RVFLs neural networks on regression problems. Through the comprehensive evaluation on 29 different UCI datasets, we show that the performances of both dRVFL and edRVFL are significantly better than the RVFL variant (OPE-RVFL) in [1] and on par with backpropagation based Deep Neural Network. Furthermore, we identify Sigmoid as the most suitable activation function for regression tasks. Finally, we propose three new deep RVFLs variants (Deep Boosted dRVFL, edRVFL+ and Two-stage edRVFL+R) which show significant improvement over the vanilla deep RVFLs. Two-stage edRVFL+R is identified as the best performing algorithm among the three newly proposed variants as it performs better than the deep neural network in 22 out of the 29 UCI datasets. The robustness of the Two-stage edRVFL+R is further confirmed by the fact that it outperforms another method (Parallel metaheuristic-ensemble of heterogeneous feedforward neural networks ) proposed in [2] by a large margin in 11 out of 14 selected datasets.