Please use this identifier to cite or link to this item: https://hdl.handle.net/10356/163261
Title: Deep metric based feature engineering to Improve document-level representation for document clustering
Authors: Xu, Liwen
Keywords: Engineering::Computer science and engineering::Computing methodologies::Document and text processing
Issue Date: 2022
Publisher: Nanyang Technological University
Source: Xu, L. (2022). Deep metric based feature engineering to Improve document-level representation for document clustering. Master's thesis, Nanyang Technological University, Singapore. https://hdl.handle.net/10356/163261
Abstract: Document-level representation attracts more and more research attention. Recent Transformer-based pretrained language models (PLMs) like BERT learn powerful textual representations. These models are originally and inherently designed for word-level tasks, which limits their maximum input length. Current document-level approaches accommodate this limitation through various ways. Some of them consider the concatenation of the title and the abstract only as the input to the PLM, which neglects the rich inherent semantic information within the main page. Other approaches try to obtain document-level representations by encoding multiple sentences in a document and concatenating them directly. However, the acquired representation may be too redundant, and the training and inference process are computationally heavy for real-world applications. To alleviate the two drawbacks, we decompose the process from word-level to document-level into a two-stage feature engineering. In the first stage, the sentence-level representations of each sentence in a document is extracted by a PLM from word-level tokens. Then they are concatenated into a document matrix. In the second stage, document matrixs with the semantic information of all text within documents are fed into a CNN model to obtain document-level representations with the dimension reduced 24 times. The model is optimized by a deep metric representation learning objective. Extensive experiments are conducted for hyper-parameter tuning and model design, and for the comparison among different deep metric representation learning objectives.
URI: https://hdl.handle.net/10356/163261
Fulltext Permission: restricted
Fulltext Availability: With Fulltext
Appears in Collections:EEE Theses

Files in This Item:
File Description SizeFormat 
Dissertation_Xu_Liwen.pdf
  Restricted Access
6.22 MBAdobe PDFView/Open

Page view(s)

54
Updated on Jan 29, 2023

Download(s)

8
Updated on Jan 29, 2023

Google ScholarTM

Check

Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.