Please use this identifier to cite or link to this item:
Title: Multi-modal deception detection in videos
Authors: Jin, Yibing
Keywords: Engineering::Electrical and electronic engineering
Issue Date: 2022
Publisher: Nanyang Technological University
Source: Jin, Y. (2022). Multi-modal deception detection in videos. Final Year Project (FYP), Nanyang Technological University, Singapore.
Abstract: Deception detection has much significance as it has many real-world applications. This project focuses on the verbal and visual modalities for deception detection from videos. The experiments were conducted on the most widely used dataset: Real-Life Trial. In the project, text information, visual information, and multimodal cues were considered individually for detecting deception. Moreover, this project used both machine learning and deep learning methods to obtain the best performance on this task. For verbal feature extraction, TF-IDF, N-Grams, and LIWC were used to transfer the text into vectors. These vectors were processed by SVM, Naïve Bayes, Random Forest, and RNN. For visual feature extraction, facial action features and gaze direction features were extracted by OpenFace. The visual features are learned by a machine learning method, SVM. This paper also used a hybrid classification model based on CNN and GRU neural networks. For the multimodal machine learning method, the features from two different modalities were concatenated together after extraction, and then the extracted features were put into SVM for classification. The experimental results suggest that the hybrid model of CNN and GRU performs best among all the methods when only information from one modality is used, and using SVM outperforms other models when two modalities are used.
Schools: School of Electrical and Electronic Engineering 
Organisations: Wuhan University
Fulltext Permission: restricted
Fulltext Availability: With Fulltext
Appears in Collections:EEE Student Reports (FYP/IA/PA/PI)

Files in This Item:
File Description SizeFormat 
FYP final report.pdf
  Restricted Access
3.08 MBAdobe PDFView/Open

Page view(s)

Updated on Feb 22, 2024

Download(s) 50

Updated on Feb 22, 2024

Google ScholarTM


Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.