Layer-wise learning framework for deep networks

Abstract

With the increasingly extensive application of deep learning, research based on experiential results has made significant progress in the field of machine learning over the past few years. However, deep learning is very difficult to understand due to its use of artificial neural networks and a black-box approach. A lack of knowledge about deep learning networks will hinder their development in situations where taking large risks is necessary, as well as restrict their use in situations where robust, dependable artificial intelligence is desired. The aim of this dissertation is to use the stochastic gradient descent method, but hierarchically, to train deep residual neural networks for better understanding. Firstly, it is of great importance to construct a mathematical model for deep residual neural networks based on matrix forms and then hierarchically train and test the deep residual neural networks on a few common image datasets employing the stochastic gradient descent technique. The case examples demonstrate a rational compromise regarding layer-wise trainability and precision while validating the applicability of the proposed layer-wise learning method to determine the optimal number of layers for real-world scenarios.