Study of convergence using residual links in various or combinations of different DNN

Abstract

Residual Network (ResNet) has gained considerable amount of attention in recent years as it has not only solve the optimisation problem in very deep network through the use of skip connections, it has also achieved state-of-the-art performance on ImageNet classification challenge.

The objective of this project is to provide insights on the residual link on convergence, how the skip connections solved the degradation problem. Of which, we found out that the degradation problem is negated as a result of skip connections as it always ensure the flow of gradients to the next residual block.

Other sub-tasks includes looking into the various variants of the ResNet architecture such as a wider and shallower ResNet and also, the applications that adopts the residual framework.