On robustness and interpretability in convolutional neural networks

Abstract

Convolutional Neural Networks achieve good performance on vision-related tasks, but often suffer from limited interpretability and are vulnerable to adversarial attacks. Past studies have shown a connection between these issues, and this work aims to quantify how bolstering robustness may influence interpretability in greater detail. We train models on the CIFAR-10 and ImageNet classification tasks to varying degrees of robustness - measured with adversarial accuracy under a fixed threat model - and evaluate their interpretability using metrics that assess various desirable aspects of model explanations. Our findings indicate that adversarial training generally yields feature attributions that are more concise and stable, though not necessarily more relevant. Finally, we propose that even modest adversarial training can meaningfully improve the quality of gradient-based explanations, but the trade-off with standard accuracy necessitates careful balancing to avoid misleading but plausible interpretations.