Machine learning for image and video summarization

Abstract

With the digital evolution of the information, the interaction with the digital display has been studied and applied in fields ranging from text entry, mouse controlling, and to online learning, human-computer interaction. The study of gaze tracking is the central part of the research regarding the interaction with the digital display as the gaze is the fastest way of showing interest on a subject. Current gazing tracking systems implement various machine learning methods such as Neural Networks, Gaussian process regression, Ensemble of Regression Trees for landmark detection and head pose estimation. However, there is no robust solution as most of the systems are still subject to limitations, including unsatisfied accuracy, significant head movement, expensive geometric setups, inconsistent lighting conditions and cumbersome calibrations. In this way, there is not enough robustness for real-world applications. Besides, while most existing gaze tracking system focuses only on estimating the gaze direction, more efforts are needed for studying the gaze tracking on a digital display. This project studies gaze tracking on a digital display with a webcam camera through a machine learning approach. Different functions, including facial landmark detection, head pose estimation, gaze projection and image processing, are studied and integrated to realize the purpose of tracking gaze on the digital display. The project was to design a gaze tracking system that provides accurate performance on a digital display that is applicable for analysis of students’ behaviors during the E-learning process.