3D human reconstruction for monitoring and predicting rehab therapeutic exercise

Abstract

Stroke is a debilitating condition that can result in hemiplegia, a type of paralysis that affects one side of the body. However, evaluating the recovery of hemiplegic patients can be a complex process as existing assessment methods are often subjective and prone to errors. This project presents the development of a camera-based human reconstruction system to objectively assess the motor functioning of hemiplegic patients. The proposed system utilizes four Azure Kinect cameras and a workstation to capture and reconstruct 3D point clouds of patients. The cameras were calibrated to obtain intrinsic and extrinsic parameters, which were used to reconstruct patients in 3D space with high accuracy. The resulting human model provides a detailed representation of the patient’s body pose and joint position. The human point cloud and skeleton obtained by body tracking enable therapists to review the patient’s movements from any angle, leading to more accurate assessments of their motor function. The system also facilitates the preservation of patient data, enabling a comparison of the patient’s motor function before and after rehabilitation. The future work aims to integrate high-accuracy and real-time machine learning models into the system, enabling more accurate human models, automatic extraction of patient limb movements, and the scoring of hemiplegia upper extremity function through an auto-assessment algorithm. This would ultimately enhance the effectiveness and efficiency of stroke rehabilitation programs by automating rehabilitation exercises and assessments.