Face super-resolution with large pose variation

Abstract

Recently many face super-resolution (FSR) algorithms have achieved great progress. The current interest lies in utilizing natural image priors from state of the art pre-trained Generative Adversarial Networks (GAN) to improve the restoration quality. With these pre-trained GAN priors, modern FSR works can recover rich texture and fine detail super-resolution (SR) face output from low resolution (LR) face input. However, while most FSR researchers focus on frontal or semi-frontal face super-resolution, the FSR performance on large face pose always be left out. To address this situation, I performed an analysis on a novel FSR algorithm - Generative Latent Bank (GLEAN) and found out that the performance of GLEAN face declined when the LR face input has a large face pose. Furthermore, through fine-tuning GLEAN with large face pose training data, I discovered GLEAN is able to learn and recover facial detail and texture of large face pose despite having GAN priors that are mostly trained on frontal faces. On top of that, I proposed a new cropped face dataset that contains over 200k+ images for evaluating FSR performance to encourage the community to focus on a small scale, extreme pose and heavily corrupted old face super-resolution problems. Finally, I concluded with future improvement directions on the proposed fine-tuned GLEAN.