Please use this identifier to cite or link to this item:
Full metadata record
DC FieldValueLanguage
dc.contributor.authorGao, Weien_US
dc.contributor.authorLiao, Guibiaoen_US
dc.contributor.authorMa, Siweien_US
dc.contributor.authorLi, Geen_US
dc.contributor.authorLiang, Yongshengen_US
dc.contributor.authorLin, Weisien_US
dc.identifier.citationGao, W., Liao, G., Ma, S., Li, G., Liang, Y. & Lin, W. (2021). Unified information fusion network for multi-modal RGB-D and RGB-T salient object detection. IEEE Transactions On Circuits and Systems for Video Technology.
dc.description.abstractThe use of complementary information, namely depth or thermal information, has shown its benefits to salient object detection (SOD) during recent years. However, the RGB-D or RGB-T SOD problems are currently only solved independently, and most of them directly extract and fuse raw features from backbones. Such methods can be easily restricted by low-quality modality data and redundant cross-modal features. In this work, a unified end-to-end framework is designed to simultaneously analyze RGB-D and RGB-T SOD tasks. Specifically, to effectively tackle multi-modal features, we propose a novel multi-stage and multi-scale fusion network (MMNet), which consists of a crossmodal multi-stage fusion module (CMFM) and a bi-directional multi-scale decoder (BMD). Similar to the visual color stage doctrine in the human visual system (HVS), the proposed CMFM aims to explore important feature representations in feature response stage, and integrate them into cross-modal features in adversarial combination stage. Moreover, the proposed BMD learns the combination of multi-level cross-modal fused features to capture both local and global information of salient objects, and can further boost the multi-modal SOD performance. The proposed unified cross-modality feature analysis framework based on two-stage and multi-scale information fusion can be used for diverse multi-modal SOD tasks. Comprehensive experiments (∼92K image-pairs) demonstrate that the proposed method consistently outperforms the other 21 state-of-the-art methods on nine benchmark datasets. This validates that our proposed method can work well on diverse multi-modal SOD tasks with good generalization and robustness, and provides a good multimodal SOD benchmark.en_US
dc.relation.ispartofIEEE Transactions on Circuits and Systems for Video Technologyen_US
dc.rights© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. The published version is available at:
dc.subjectEngineering::Computer science and engineeringen_US
dc.titleUnified information fusion network for multi-modal RGB-D and RGB-T salient object detectionen_US
dc.typeJournal Articleen
dc.contributor.schoolSchool of Computer Science and Engineeringen_US
dc.description.versionAccepted versionen_US
dc.subject.keywordsDynamic Cross-Modal Guided Mechanismen_US
dc.subject.keywordsRGBD/RGB-T Multi-Modal Dataen_US
dc.description.acknowledgementThis work was supported by Ministry of Science and Technology of China - Science and Technology Innovations 2030 (2019AAA0103501), Natural Science Foundation of China (61801303 and 62031013), Guangdong Basic and Applied Basic Research Foundation (2019A1515012031), and Shenzhen Science and Technology Plan Basic Research Project (JCYJ20190808161805519).en_US
item.fulltextWith Fulltext-
Appears in Collections:SCSE Journal Articles

Page view(s)

Updated on Jul 4, 2022

Download(s) 50

Updated on Jul 4, 2022

Google ScholarTM




Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.