Please use this identifier to cite or link to this item:
Title: Multimodal knowledge-based analysis in multimedia event detection
Authors: Younessian, Ehsan.
Mitamura, Teruko.
Hauptmann, Alexander.
Issue Date: 2012
Source: Younessian, E., Mitamura, T.,& Hauptmann, A. (2012). Multimodal knowledge-based analysis in multimedia event detection. Proceedings of the 2nd ACM International Conference on Multimedia Retrieval - ICMR '12.
Abstract: Multimedia Event Detection (MED) is a multimedia retrieval task with the goal of finding videos of a particular event in a large-scale Internet video archive, given example videos and text descriptions. We focus on the multimodal knowledge-based analysis in MED where we utilize meaningful and semantic features such as Automatic Speech Recognition (ASR) transcripts, acoustic concept indexing (i.e. 42 acoustic concepts) and visual semantic indexing (i.e. 346 visual concepts) to characterize videos in archive. We study two scenarios where we either do or do not use the provided example videos. In the former, we propose a novel Adaptive Semantic Similarity (ASS) to measure textual similarity between ASR transcripts of videos. We also incorporate acoustic concept indexing and classification to retrieve test videos, specially with too few spoken words. In the latter 'ad-hoc' scenario where we do not have any example video, we use only the event kit description to retrieve test videos ASR transcripts and visual semantics. We also propose an event-specific fusion scheme to combine textual and visual retrieval outputs. Our results show the effectiveness of the proposed ASS and acoustic concept indexing methods and their complimentary role. We also conduct a set of experiments to assess the proposed framework for the 'ad-hoc' scenario.
DOI: 10.1145/2324796.2324855
Fulltext Permission: none
Fulltext Availability: No Fulltext
Appears in Collections:SCSE Conference Papers

Citations 10

Updated on Mar 10, 2021

Page view(s) 50

Updated on Jun 27, 2022

Google ScholarTM




Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.