Semantics-aware visual object tracking

Abstract

In this paper, we propose a semantics-aware visual object tracking method, which introduces semantics into the tracking procedure and extends the model of an object with explicit semantics prior to enhancing the robustness of three key aspects of the tracking framework, i.e., appearance model, search scheme, and scale adaptation. We first present a semantic object proposal generation method for video sequences to generate high-quality category-oriented object proposals. Then, a hybrid semantics-aware tracking algorithm with semantic compatibility is proposed. This algorithm takes full advantages of globally sparse semantic object proposal prediction and locally dense prediction with a template model and semantic distractor-aware color appearance model. Furthermore, we propose to exploit semantics to localize object accurately via an energy minimization framework-based scale adaptation method, which jointly integrates dense location prior, instance-specific color, and category-specific semantic information. Extensive experiments are conducted on two widely used benchmarks, and the results demonstrate that our method achieves the state-of-the-art performance.