Please use this identifier to cite or link to this item:
Title: Robust-EQA: robust learning for embodied question answering with noisy labels
Authors: Luo, Haonan
Lin, Guosheng
Shen, Fumin
Huang, Xingguo
Yao, Yazhou
Shen, Hengtao
Keywords: Engineering::Computer science and engineering
Issue Date: 2023
Source: Luo, H., Lin, G., Shen, F., Huang, X., Yao, Y. & Shen, H. (2023). Robust-EQA: robust learning for embodied question answering with noisy labels. IEEE Transactions On Neural Networks and Learning Systems.
Project: AISGRP-2018-003
Journal: IEEE Transactions on Neural Networks and Learning Systems
Abstract: Embodied question answering (EQA) is a recently emerged research field in which an agent is asked to answer the user's questions by exploring the environment and collecting visual information. Plenty of researchers turn their attention to the EQA field due to its broad potential application areas, such as in-home robots, self-driven mobile, and personal assistants. High-level visual tasks, such as EQA, are susceptible to noisy inputs, because they have complex reasoning processes. Before the profits of the EQA field can be applied to practical applications, good robustness against label noise needs to be equipped. To tackle this problem, we propose a novel label noise-robust learning algorithm for the EQA task. First, a joint training co-regularization noise-robust learning method is proposed for noisy filtering of the visual question answering (VQA) module, which trains two parallel network branches by one loss function. Then, a two-stage hierarchical robust learning algorithm is proposed to filter out noisy navigation labels in both trajectory level and action level. Finally, by taking purified labels as inputs, a joint robust learning mechanism is given to coordinate the work of the whole EQA system. Empirical results demonstrate that, under extremely noisy environments (45% of noisy labels) and low-level noisy environments (20% of noisy labels), the robustness of deep learning models trained by our algorithm is superior to the existing EQA models in noisy environments.
ISSN: 2162-237X
DOI: 10.1109/TNNLS.2023.3251984
Schools: School of Computer Science and Engineering 
Rights: © 2023 IEEE. All rights reserved.
Fulltext Permission: none
Fulltext Availability: No Fulltext
Appears in Collections:SCSE Journal Articles

Citations 50

Updated on Nov 30, 2023

Web of ScienceTM
Citations 50

Updated on Oct 28, 2023

Page view(s)

Updated on Dec 3, 2023

Google ScholarTM




Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.