Deep learning for humanlike character motion control in VR table tennis

Abstract

In character motion control, reinforcement learning (RL) has provided new methods to create controllers for simulated characters. The latest research gives us a framework that creates controllers that are both humanlike and dynamic, which solves the initial problem of these RL based controllers. This framework however is only implemented using the Bullet physics engine, and thus cannot be directly implemented in game engines that use different physics engines, such as the Unity game engine. This project aims to improve on an adaptation of this framework on Unity by implementing it on a VR tennis game. The agent tries to mimic a given action and react to the environment around it. The different RL policies available are compared, and limitations of this adaptation on Unity are discussed as well.