Please use this identifier to cite or link to this item:
Title: Adversarial training using meta-learning for BERT
Authors: Low, Timothy Jing Haen
Keywords: Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence
Issue Date: 2022
Publisher: Nanyang Technological University
Source: Low, T. J. H. (2022). Adversarial training using meta-learning for BERT. Final Year Project (FYP), Nanyang Technological University, Singapore.
Project: SCSE21-0524
Abstract: Deep learning is currently the most successful method of semantic analysis in natural language processing. However, in recent years, many variants of carefully crafted inputs designed to cause misclassification, known as adversarial attacks, have been engineered with tremendous success. One well-known, efficient method to develop models to be robust against adversarial attacks is known as adversarial training, where models are iteratively trained on samples produces by the specific attack algorithm. However, adversarial training only works when the model has access to the attack generation algorithm or a large dataset of attack samples, and so cannot defend against attacks of which they have access to a low number of samples. This project proposes to overcome this challenge using meta-learning, which uses a large number of similar tasks from a different domain to train a classifier to learn another task for which a small number of labelled samples are available. We show that by using the Model-Agnostic Meta-Learning algorithm in adversarial training, a model trained on a large number of different adversarial attacks can become more robust to an adversarial attack that it has few samples of. This project will also explore augmenting the training set with a large number of non-adversarial perturbations, in order to possibly better mitigate adversarial attacks
Schools: School of Computer Science and Engineering 
Fulltext Permission: restricted
Fulltext Availability: With Fulltext
Appears in Collections:SCSE Student Reports (FYP/IA/PA/PI)

Files in This Item:
File Description SizeFormat 
Final Year Project_Timothy Low Jing Haen_U1820644B.pdf
  Restricted Access
820.09 kBAdobe PDFView/Open

Page view(s)

Updated on Dec 9, 2023


Updated on Dec 9, 2023

Google ScholarTM


Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.