A Ulam's game for video based action recognition

Abstract

In this dissertation, we propose a conditional early exiting framework with Ulam’s Game for action recognition. Since the action recognition system has extremely high requirements on dynamic performance, our system pays more attention to improving the detection efficiency of the system, hoping to obtain the detection results in a shorter time. In our system, we use a modified ResNet-50 as backbone network to do feature extraction and use a Pooling module to accumulate feature. Then, we have a neural network Gate module to determine whether the feature have accumulated enough to begin Ulam’s Game. A classifier is used to get candidate results, which are used to run Ulam’s Game and get the final prediction. The model shows good detection accuracy and dynamic performance in multiple data sets (Mini-Kinetics, ActivityNet).