Automatic assessment of body motion and pose imitation

Abstract

The proliferation of social media in recent years have allowed individuals to learn and practice dance using online visual guides, at their own pace and space. However, the lack of concrete feedback will pose a challenge to them as they need to be their own judge. To help the increasing number of individual learners, this project aims to develop a system that can generate a similarity score of the learner’s dance movements and provide them with personalized feedback on areas to improve on. The system would leverage on pose estimation technologies to extract the pose of the learner and their instructor frame-by-frame. By aligning the frames between the learner’s and instructor’s pose sequence, the accuracy in each frame is calculated to generate an overall similarity score between the videos. According to the accuracy of the movement in each frame, the movements that require improvement and movements that have been executed well would be highlighted differently in the feedback video. Various methods were implemented and evaluated against a user study, to identify the most effective approach.