Privacy-preserving image super-resolution

Abstract

The objective of the study below is to explore the use of Single Image Super-Resolution (SISR) with an adversarial component included in the training process to fool facial recognition (FR) models into misidentifying faces. This study applies the use of modern SISR techniques such as the use of a Swin Transformer-based model, SwinIR to study its effectiveness in fooling a fixed FR model FaceNet. To investigate the practicality of this architecture, a weighted loss function consisting of MAE, Perceptual Loss, SSIM Loss and Adversarial loss components was used to train the SwinIR model. The results were promising, with the fooling rate of the SR improving over the course of the training. Although these results show the potential of using transformer-based models for privacy preservation, there are limitations to the study, such as using only the limited and biased CelebA dataset in training and relying on pre-trained models such as SwinIR and FaceNet. A more robust and comprehensive approach could be explored in the future, by using or creating new datasets for training, using a specialized SR model as the baseline for training, and more extensive compute resources. This is, to the best of our knowledge, the first time the use of transformer-based SR models to preserve privacy against FR models has been explored. This can potentially help to preserve privacy of users of online photo-sharing platforms, which have been used to identify people without consent.