An empirical study of the inherent resistance of knowledge distillation based federated learning to targeted poisoning attacks

Abstract

While the integration of Knowledge Distillation (KD) into Federated Learning (FL) has recently emerged as a promising solution to address the challenges of heterogeneity and communication efficiency, little is known about the security of these schemes against poisoning attacks prevalent in vanilla FL. From recent countermeasures built around KD, we conjecture that the way knowledge is distilled from the global model to the local models and the type of knowledge transfer by KD themselves offer some resilience against targeted poisoning attacks in FL. To attest this hypothesis, we systematize various adversary agnostic state-of-the-art KD-based FL algorithms for the evaluation of their resistance to different targeted poisoning attacks on two vision recognition tasks. Our empirical security-utility trade-off study indicates surprisingly good inherent immunity of certain KD-based FL algorithms that are not designed to mitigate these attacks. By probing into the causes of their robustness, the KD space exploration provides further insights into the balancing of security, privacy and efficiency triad in different FL settings.