Detecting gaze interaction in video sequences

Abstract

The research about exploring the interaction of human action has been popular in the field of computer vision. Specially, detecting whether people are looking at each other offers great help to figure out the relationship between people in video sequences. In this dissertation, our first method is LAEO-Net with slightly change to fit in our dataset’s meeting environment, including head detection, track of head position and final bin-classification. It is made up of three branches, two of them employ 3D convolutional layers to extract the characteristic and the other applies 2D convolutional layers for head relative position. Besides, we propose another method to do the auxiliary determination. It can be divided in three different mathematical models based on the multi-loss head pose estimation. Three Euler angels presenting head pose are obtained by the network separately, in order to improve the accuracy of the final results. In the part of evaluation, we choose UCO-LAEO and AVA-LAEO datasets to do the training and compare above methods on TVHID dataset and do the analysis according to respective advantages and weakness.