Understanding variations (variant & invariant) of classification tasks/targets

Abstract

There still lacks a certain mechanism to cater for variance in data and a lack of levels of impact brought by variance. We introduce a composite term called learning, where average improvement upon every epoch divided by previous loss value to have a standard reference across our models of differing architecture. We use specially designed datasets on Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) to observe the effects of variance on bottom-up and top-down neural network architectures respectively. We find that variance has degrees, such that given datasets of different applied operations, the amount of loss varies notably. We find that variance has dimensions, such that the amount of variance introduced to the image, affects the confidence of the model prediction. We find that even providing a single training data with no operation applied to it, the CNN and RNN architecture could give lower validation losses (with CNN being significantly lower). This study shows the significance of variance impact on model performance manifested in data and the pressing need to understand variance to better design mitigations and mechanisms to handle it.