Development of learning algorithms for fully connect layers of convolutional neural network

Abstract

In the field of transfer learning, pre-trained weights are usually used in the Convolutional Layers to get the features of the data and Fully Connected Layers are trained again for another classification task based on new training data.

Inspired by the ideas of transfer learning and kinematic control algorithm in the field of robotics, this dissertation proposes a new learning algorithm which can be used to train the Fully Connected Layers in a simple way.

It is observed based on the testing results of Cifar-10 that the algorithm is able to achieve a better accuracy at the start of the training while the accuracy of baseline method, Stochastic Gradient Descent, is lower at the beginning. The accuracy of the algorithm improves and becomes stable in the progress of learning.

Thus, the algorithm has the potential of achieving a better performance in lesser epochs of training as compared with traditional Gradient Descent Method.