Deep learning architecture analysis with mapper

Abstract

In recent years, we have seen the rise of deep learning models such as convolutional neural networks (CNN) for image classification. However, we do not understand what makes these networks able to achieve such outstanding performance. Upon building a neural network, we can only see the input and output of the model and the in-between remains as a mystery or a black box to us. In this paper, given a trained deep neural network, we address the interpretability issue by probing neuron activation. We use a tool in topological data analysis (TDA), known as mapper, to visualize relationships between different activation in a particular layer of the specified neural network. Mapper provides two topological summaries, namely branches and loops. The effectiveness of mapper depends on the dataset being used. In the case of image classification tasks, if the images are dissimilar, mapper can construct informative branches to visualize the relationships between activation. However, if the images are very similar, mapper is not useful. For tabular data, mapper is useful only if the majority of the features are continuous variables, as demonstrated by the Iris dataset example. For text data, the usefulness of mapper in visualizing the activation is determined by the length and content of the text. If the text is short and focuses on the same content, mapper is not useful for visualizing the activation.