Unsupervised feature learning with sparse Bayesian auto-encoding based extreme learning machine

Abstract

Extreme learning machine (ELM) is a popular method in machine learning with extremely few parameters, fast learning speed and model efficiency. Unsupervised feature learning based ELM receives rising research focus. Recently the ELM auto-encoder (ELM-AE) was proposed for this task, which develops the ELM based compact feature learning without sacrificing elegant solution. Compared with ELM-AE and following ℓ1-regularized ELM-AE, we introduce a sparse Bayesian learning scheme into ELM-AE for better generalization capability. A parallel training strategy is also integrated to improve time-efficiency of multi-output sparse Bayesian learning. Furthermore, pruning hidden nodes for better performance and efficiency according to estimated variances of prior distribution of output weights is achieved. Experiments on several datasets verify the effectiveness and efficiency of our proposed ELM-AE for unsupervised feature learning, compared with PCA, NMF, ELM-AE and ℓ1-regularized ELM-AE.