Head movement mimic - video-based human head motion transfer

Abstract

In an era where deepfake technology continues to evolve, human head motion transfer represents an innovative application with vast potential in entertainment and virtual reality. This project aims to develop a web application that leverages the Thin-PlateSpline (TPS) Motion Model, Motion Refinement for Unsupervised Face Animation (MRFA) and Facial Scene Representation Transformer (FSRT) to map head movements from a driving video onto a static image, generating a realistic mimicked video. By incorporating principles of Human-Computer Interaction (HCI), the application is designed for ease of use and accessibility, allowing non-technical users to engage with cutting-edge technologies. This report discusses the system’s design, implementation, and user studies conducted to evaluate usability. Additionally, the project integrated the models, and evaluated usability of the application through testing with a diverse set of participants. The findings demonstrated the potential of these models in motion transfer applications and the importance of balancing technical performance with user experience.