PoseInpaint: pose-based face data augmentation with prompt-driven image inpainting

Abstract

This paper introduces PoseInpaint, a novel face data augmentation pipeline that enhances dataset generalisation to real-world conditions by fusing three stages: face alignment and pose synthesis, segmentation mask generation, and prompt-driven image inpainting. The approach enables the generation of pose-augmented face images with controlled occlusions, achieving performance improvements in face recognition and verification tasks. Ablation studies using PoseInpaint demonstrate up to 22.48% and 35.91% improvements in Top-1 accuracy and True Acceptance Rate (TAR), respectively, over both the baseline and standalone augmentation methods. Generalisation experiments show consistent improvements in cosine similarity scores between training and unseen occluded test sets. PoseInpaint provides a robust and effective solution for advancing face data augmentation, offering significant gains in recognition and verification performance.