Highly controllable motion generation model

Abstract

Text-To-Motion generation has emerged as a promising area of research in deep learning, with potential applications in video games, animation and virtual reality systems. However, the adoption of these technologies is still limited due to the predefined skeletal prior. Thus, manual effort is required to rig the desired target meshes with a compatible skeleton. On the other hand, recent advancements in 3D-Human generation have demonstrated the capability to produce detailed and realistic 3D character models from textual inputs. The gap between motion and 3D-human generation is a compelling area of research. A pipeline that can automate the transfer of motion to generated 3D- human model will significantly simplify the workflow of generating 3D animations for the laypersons. This project reviews the state-of-the-art (SOTA) approaches in motion and 3D-Human generation, as well as methods in ensuring seamless compatibility between them. We propose a pipeline that integrates both models to enable automated and user-friendly workflows for creating 3D animations whilst ensuring compatibility with popular 3D software platforms like Unreal Engine and Blender.