Evaluation of backdoor attacks on deep neural networks

Abstract

Deep Neural Networks (DNNs) are increasingly deployed for use in critical applications in organizations everywhere, but yet the time and cost complexity of a state-of-the-art model can mean outsourcing parts of the DNN model training, either through third-party datasets or models, opening themselves up to backdoor attacks such as data poisoning.

This study evaluates the effectiveness of BadNet attacks across multiple DNN models (e.g., WideResNet50, MobileNetV3-Large) and datasets (CIFAR10, TinyImageNet) under real-world constraints: black-box access, non-targeted attacks, and dataset-only manipulation. We analyse key attack parameters—poisoning ratio and trigger size—and their impact on a backdoored model’s clean accuracy (ACC) and attack success rate (ASR). Our findings reveal that complex datasets (e.g., TinyImageNet) are more vulnerable, requiring lower poisoning ratios (0.5–2%) for high ASR, while simpler datasets (e.g., CIFAR10) demand higher ratios. Smaller and less complex models (e.g., MobileNetV3-Large) are more susceptible, achieving 100% ASR with minimal poisoning.

We further assess three defences—Anti-Backdoor Learning (ABL), Channel Lipschitz Pruning (CLP), and Neural Attention Distillation (NAD)—and demonstrate that NAD is most effective for complex datasets, and CLP performs better on simpler models. However, no single defence universally mitigates all attack configurations. Our results highlight the need for risk-avoidant model selection, dataset verification, and selecting appropriate defences to protect against backdoor threats.