Spatio-temporal enhanced sparse feature selection for video saliency estimation

Abstract

Video saliency mechanism is crucial in the human visual system and helpful to object detection and recognition. In this paper we propose a novel video saliency model that video saliency should be both consistently salient among consecutive frames and temporally novel due to motion or appearance changes. Based on the model, temporal coherence, in addition to spatial saliency, is fully considered by introducing temporal consistence and temporal difference into sparse feature selections. Features selected spatio-temporally are enhanced and fused together to generate the proposed video saliency maps. Comparisons with several state-of-th-art methods on two public video datasets further demonstrate the effectiveness of our method.