Automatic video genre classification with visual words

Abstract

Automated content analysis has been growing popular in the research field given the vast and increasing amount of digital content. Content analysis is applicable in many areas including content management, searching and browsing. It is going to transcend the need to manually process digital content. One of the promising topics is automatic video content classification. Numerous research works have been done on this topic. The result, however, have not been very attractive. This project aims to develop a reliable framework to automatically classify content of a video stream. It proposes to apply bag-of-words, a well-known method in text processing literature to the problem of video content classification. Recently this method has received attention in some problem domain such as object retrieval. Bag-of-words characterizes a text document by occurrences of different words and their frequencies of occurrence. This project builds the visual analogy of word and represents visual documents based on this analogy. Text classification techniques are then applied. Two major visual features, Scale-Invariant Feature Transform (SIFT) and Gabor, are evaluated in implementing bag-of-words. The implementation with SIFT is found to be more robust. Bag-of-words’ performance is also empirically proven to be more effective than the alternative of using global Gabor method. An automatic video genre classification framework is developed based on these results. Its scope is limited to sport videos. Four genres are experimented with: football, basketball, golf and tennis. The classification result is very promising. The overall accuracy rate is 91 percent. The algorithm’s speed, however, still needs to be further improved.