Please use this identifier to cite or link to this item:
Title: Early action recognition with category exclusion using policy-based reinforcement learning
Authors: Weng, Junwu
Jiang, Xudong
Zheng, Wei-Long
Yuan, Junsong
Keywords: Engineering::Electrical and electronic engineering
Issue Date: 2020
Source: Weng, J., Jiang, X., Zheng, W.-L., & Yuan, J. (2019). Early action recognition with category exclusion using policy-based reinforcement learning. IEEE Transactions on Circuits and Systems for Video Technology, in-press. doi:10.1109/TCSVT.2020.2976789
Journal: IEEE Transactions on Circuits and Systems for Video Technology
Abstract: The goal of early action recognition is to predict action label when the sequence is partially observed. The existing methods treat the early action recognition task as sequential classification problems on different observation ratios of an action sequence. Since these models are trained by differentiating positive category from all negative classes, the diverse information of different negative categories is ignored, which we believe can be collected to help improve the recognition performance. In this paper, we step towards to a new direction by introducing category exclusion to early action recognition. We model the exclusion as a mask operation on the classification probability output of a pre-trained early action recognition classifier. Specifically, we use policy-based reinforcement learning to train an agent. The agent generates a series of binary masks to exclude interfering negative categories during action execution and hence help improve the recognition accuracy. The proposed method is evaluated on three benchmark recognition datasets, NTU-RGBD, First-Person Hand Action, as well as UCF-101. The proposed method enhances the recognition accuracy consistently over all different observation ratios on the three datasets, where the accuracy improvements on the early stages are especially significant.
ISSN: 1051-8215
DOI: 10.1109/TCSVT.2020.2976789
Rights: © 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. The published version is available at:
Fulltext Permission: open
Fulltext Availability: With Fulltext
Appears in Collections:IMI Journal Articles

Page view(s)

Updated on Jan 17, 2021


Updated on Jan 17, 2021

Google ScholarTM




Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.