Please use this identifier to cite or link to this item: https://hdl.handle.net/10356/165534
Title: Abstractive summarization framework based on pre-training and contrastive learning
Authors: Yi, Chenqi
Keywords: Engineering::Electrical and electronic engineering
Issue Date: 2023
Publisher: Nanyang Technological University
Source: Yi, C. (2023). Abstractive summarization framework based on pre-training and contrastive learning. Master's thesis, Nanyang Technological University, Singapore. https://hdl.handle.net/10356/165534
Abstract: Abstractive summarization aims at generating sentences which can well cover the key information of the document. In this dissertation, we verify the effectiveness of a generation-evaluation model trained with contrastive learning, which generates a set of candidate summaries first and then evaluates the candidates to select the best one. Conventional methods directly introduce pre-trained models by default as the backbone of summary evaluation model. However, what pre-training task is helpful for improving the performance of pre-trained models on downstream summary evaluation task is still an open question. We conduct a study on Inverse Cloze Task (ICT) to answer the question. For the backbone of evaluation model, we compare the results of different pre-trained models. We further adopt ICT as additional pre-training task to pre-train the model and utilize it as the backbone of the evaluation model. We also verify and analyze how the masking rate in ICT affects the downstream evaluation task. Experiments on XSum and CNN/Daily Mail show that the model with additional ICT pre-training outperforms other pre-training baselines.
URI: https://hdl.handle.net/10356/165534
Schools: School of Electrical and Electronic Engineering 
Fulltext Permission: restricted
Fulltext Availability: With Fulltext
Appears in Collections:EEE Theses

Files in This Item:
File Description SizeFormat 
Abstractive summarization framework based on pre-training and contrastive learning.pdf
  Restricted Access
1.13 MBAdobe PDFView/Open

Page view(s)

180
Updated on May 19, 2024

Download(s)

14
Updated on May 19, 2024

Google ScholarTM

Check

Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.