Please use this identifier to cite or link to this item:
Title: Visual event recognition in videos by learning from web data
Authors: Duan, Lixin
Xu, Dong
Tsang, Ivor Wai-Hung
Luo, Jiebo
Keywords: DRNTU::Engineering::Computer science and engineering::Data
Issue Date: 2012
Source: Duan, L., Xu. D., Tsang, I. W., & Luo, J. (2012). Visual Event Recognition in Videos by Learning from Web Data. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(9), 1667-1680.
Series/Report no.: IEEE transactions on pattern analysis and machine intelligence
Abstract: We propose a visual event recognition framework for consumer videos by leveraging a large amount of loosely labeled web videos (e.g., from YouTube). Observing that consumer videos generally contain large intraclass variations within the same type of events, we first propose a new method, called Aligned Space-Time Pyramid Matching (ASTPM), to measure the distance between any two video clips. Second, we propose a new transfer learning method, referred to as Adaptive Multiple Kernel Learning (A-MKL), in order to 1) fuse the information from multiple pyramid levels and features (i.e., space-time features and static SIFT features) and 2) cope with the considerable variation in feature distributions between videos from two domains (i.e., web video domain and consumer video domain). For each pyramid level and each type of local features, we first train a set of SVM classifiers based on the combined training set from two domains by using multiple base kernels from different kernel types and parameters, which are then fused with equal weights to obtain a prelearned average classifier. In A-MKL, for each event class we learn an adapted target classifier based on multiple base kernels and the prelearned average classifiers from this event class or all the event classes by minimizing both the structural risk functional and the mismatch between data distributions of two domains. Extensive experiments demonstrate the effectiveness of our proposed framework that requires only a small number of labeled consumer videos by leveraging web data. We also conduct an in-depth investigation on various aspects of the proposed method A-MKL, such as the analysis on the combination coefficients on the prelearned classifiers, the convergence of the learning algorithm, and the performance variation by using different proportions of labeled consumer videos. Moreover, we show that A-MKL using the prelearned classifiers from all the event classes leads to better performance when compared with A-MK- using the prelearned classifiers only from each individual event class.
ISSN: 0162-8828
DOI: 10.1109/TPAMI.2011.265
Rights: © 2012 IEEE
Fulltext Permission: none
Fulltext Availability: No Fulltext
Appears in Collections:SCSE Journal Articles

Google ScholarTM




Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.