Please use this identifier to cite or link to this item:
Title: Sequential fusion of facial appearance and dynamics for depression recognition
Authors: Chen, Qian
Chaturvedi, Iti
Ji, Shaoxiong
Cambria, Erik
Keywords: Engineering::Computer science and engineering
Issue Date: 2021
Source: Chen, Q., Chaturvedi, I., Ji, S. & Cambria, E. (2021). Sequential fusion of facial appearance and dynamics for depression recognition. Pattern Recognition Letters, 150, 115-121.
Journal: Pattern Recognition Letters
Abstract: In mental health assessment, it is validated that nonverbal cues like facial expressions can be indicative of depressive disorders. Recently, the multimodal fusion of facial appearance and dynamics based on convolutional neural networks has demonstrated encouraging performance in depression analysis. However, correlation and complementarity between different visual modalities have not been well studied in prior methods. In this paper, we propose a sequential fusion method for facial depression recognition. For mining the correlated and complementary depression patterns in multimodal learning, a chained-fusion mechanism is introduced to jointly learn facial appearance and dynamics in a unified framework. We show that such sequential fusion can provide a probabilistic perspective of the model correlation and complementarity between two different data modalities for improved depression recognition. Results on a benchmark dataset show the superiority of our method against several state-of-the-art alternatives.
ISSN: 0167-8655
DOI: 10.1016/j.patrec.2021.07.005
Schools: School of Computer Science and Engineering 
Rights: © 2021 Elsevier B.V. All rights reserved.
Fulltext Permission: none
Fulltext Availability: No Fulltext
Appears in Collections:SCSE Journal Articles

Citations 20

Updated on May 30, 2023

Web of ScienceTM
Citations 20

Updated on Jun 6, 2023

Page view(s)

Updated on Jun 6, 2023

Google ScholarTM




Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.