Please use this identifier to cite or link to this item:
Title: Task-generic semantic convolutional neural network for web text-aided image classification
Authors: Wang, Dongzhe
Mao, Kezhi
Keywords: Engineering::Electrical and electronic engineering
Issue Date: 2019
Source: Wang, D. & Mao, K. (2019). Task-generic semantic convolutional neural network for web text-aided image classification. Neurocomputing, 329, 103-115.
Journal: Neurocomputing
Abstract: In this work, we explore how to use external and auxiliary web text to improve image classification. The keystone of web text-aided image classification is the representation learning for these two modalities of data. In the recent decade, convolutional neural networks (CNN) as the core representation methods of images have become a commodity in computer vision community. On the other hand, the long reign of word vectors has the same wide-ranging impact on NLP for representation learning. Based on the pre-trained word vectors, we propose a novel semantic CNN (s-CNN) model for high-level text representation learning using task-generic semantic filters. However, the s-CNN model inevitably brings about surplus semantic filters to achieve better applicability and generalization in universal tasks. Moreover, the surplus filters may lead to semantic overlaps and feature redundancy issue. To address this issue, we develop the so-called s-CNN Clustered (s-CNNC) models that uses filter clusters instead of individual filters. Interacting with the image CNN models, the s-CNNC models can further boost image classification under a multi-modal framework (mm-CNN). In addition, we propose to use the external text information selectively in the mm-CNN network to alleviate the noise problem inherent in web text. We validate the effectiveness of the proposed models on six benchmark datasets, and the results show that our approaches achieve remarkable improvements.
ISSN: 0925-2312
DOI: 10.1016/j.neucom.2018.09.042
Rights: © 2018 Elsevier B.V. All rights reserved.
Fulltext Permission: none
Fulltext Availability: No Fulltext
Appears in Collections:EEE Journal Articles

Page view(s)

Updated on Nov 28, 2021

Google ScholarTM




Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.