Please use this identifier to cite or link to this item: https://hdl.handle.net/10356/78809
Title: Enhance QANet by BERT for machine reading comprehension
Authors: Yin, Bo
Keywords: Engineering::Electrical and electronic engineering
Issue Date: 2019
Abstract: The aim of this dissertation is to study the implementation of QANet on SQuAD for the machine reading comprehension task, and enhance QANet by introducing the contextual representation BERT architecture for the task. First of all, several techniques employed to capture the dependencies among words in the question answering task are reviewed, including Recurrent Neural Network and the attention mechanism. Then a review is conducted about the heated NLP downstream task: question answering and its subclasses, which contains the most important machine reading comprehension task. It follows by a review of two categories of reading comprehension models. Then the word representation methods are introduced in detail and how they evolved across the whole time is provided. The architecture of one reading comprehension model QANet is reviewed with introduction to each layer. The most important part of this dissertation is about understanding the implementation of the QANet and finding the best performance of the original QANet by tuning hyperparameters. It follows by applying the state-of-art contextual word representation method BERT to enhance QANet. The experiment results depict that BERT improve F1 score on validation dataset by 7.2 percent. Ablation study has been conducted to verify how those model components of QANet can contribute to the overall performance of the model, such as the application of attention mechanism to capture the global interactions. The QANet with BERT model is also compared to BiDAF with BERT model to demonstrate that QANet with BERT may have better performance than another model using RNNs with BERT on some datasets.
URI: http://hdl.handle.net/10356/78809
Fulltext Permission: restricted
Fulltext Availability: With Fulltext
Appears in Collections:EEE Theses

Files in This Item:
File Description SizeFormat 
YINBO_final_signed.pdf
  Restricted Access
2 MBAdobe PDFView/Open

Page view(s) 20

128
checked on Oct 21, 2020

Download(s) 20

7
checked on Oct 21, 2020

Google ScholarTM

Check

Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.