Recognising activities using motion history

Abstract

Human Activity Recognition (HAR) is a daunting task of computer vision. Complex algorithms are required to recognize actions performed, through spatial-temporal information obtained from video sequences. Large computing power is required to process this information. Motion History Images (MHI) can represent this information in a single image, hence it can be used to reduce the complexities and hardware demands in implementing HAR. The objective of this project was to implement HAR using MHI. The implementation first involved the acquisition of video sequences. Frames from the video sequences were then pre-processed and converted into MHI for annotation and creation of dataset. A convolutional neural network (CNN) model was used to train on the dataset. The model was then validated and tested to evaluate its effectiveness, before being integrated into the HAR program. While the model performed very well for the validation set, there were mixed results for the testing set. The poorer results were due to insufficient intraclass variation in some classes in the training set, and the model responded not as well to actions that were slightly different. However, the better results demonstrate that certain actions can be recognized well in a generalized setting. In integrating the HAR program, results showed that it is unable to run in real-time due to hardware constraints, but real-time speeds are attainable, through using better computing hardware. Future works on this project include varying the conditions used for the recording of actions performed to enable the model to generalize better in HAR. The use of better computing hardware will enable the HAR program to run in real-time, and in turn deployable in real life applications.