PoseBuddy: revolutionizing fitness with real-time pose estimation and AI-driven guidance

Abstract

The integration of artificial intelligence into the fitness industry has opened new possibilities for personalized exercise coaching. PoseBuddy is an AI-driven mobile application designed to analyse exercise form in real-time, providing users with automated feedback to improve their squat technique. The system leverages MoveNet, a lightweight pose estimation model, to extract keypoints from exercise videos, followed by a machine learning classifier that determines whether the form is "good" or "bad." To enhance user experience, a large language model (LLM) generates personalized recommendations based on key movement features. This study explores the design, development, and evaluation of PoseBuddy, detailing its implementation as a cloud-based microservice architecture hosted on AWS. The classifier was trained using manually labelled squat videos, with K-Nearest Neighbors (KNN) emerging as the most effective model for form classification due to its superior recall and accuracy. The LLM component was fine-tuned using prompt engineering techniques, ensuring that feedback was both actionable and informative. User survey results demonstrated that PoseBuddy provides valuable insights, particularly for beginners and intermediate users. However, challenges remain in feedback clarity and depth of analysis. Beginners found technical explanations difficult to follow while advanced users highlighted the need for multi-angle analysis for better movement evaluation. This project highlights the potential of AI-driven fitness applications to bridge the gap between users and expert coaching, making personalised exercise guidance more accessible.