Please use this identifier to cite or link to this item: https://hdl.handle.net/10356/164525
Title: Crowdsourcing-based automated essay scoring framework
Authors: Bai, Huanyu
Keywords: Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence
Issue Date: 2022
Publisher: Nanyang Technological University
Source: Bai, H. (2022). Crowdsourcing-based automated essay scoring framework. Master's thesis, Nanyang Technological University, Singapore. https://hdl.handle.net/10356/164525
Abstract: Automated Essay Scoring (AES) is a challenging topic in Natural Language Processing. Although deep learning models achieve remarkable performance for the AES task, they have two major problems. Most AES models are unable to handle the out-of-vocabulary (OOV) words. Besides, training AES models is costly in practice. This thesis aims to handle these two problems. To tackle the OOV problem, this thesis proposes the Gated Character-aware Convolutional Neural Network (GCCNN) model. The GCCNN model incorporates character-level information into the AES model by using a vector gating mechanism to fuse the word-level and character-level information. The experimental results show that the proposed GCCNN model outperforms several strong baseline models. In addition, the qualitative analysis demonstrates the importance of character-level information for tackling the OOV problem for the AES task. Moreover, this thesis proposes the Incremental Learning with Dynamic Exemplar Herding (ILDEH) approach to efficiently train the AES models. The ILDEH approach trains the AES model in the crowdsourcing environment. To effectively improve the AES performance, the ILDEH approach simultaneously tackles catastrophic forgetting and concept drift by the Linear Outlier Suppression loss and Dynamic Exemplar Herding algorithm. The experimental results show that the ILDEH approach outperforms all baseline approaches and significantly reduces the training time. Note that the ILDEH approach is model-agnostic and can also be applied to other classification tasks. Lastly, this thesis proposes the Crowdsourcing-based Automated Essay Scoring (CAES) framework by integrating the GCCNN model and the ILDEH approach. The CAES framework collects graded essays online through crowdsourcing. The ILDEH approach is used to incrementally train the GCCNN models, which are used to provide instant essay assessment service. By using the CAES framework, one single CPU server is sufficient for most AES systems.
URI: https://hdl.handle.net/10356/164525
DOI: 10.32657/10356/164525
Schools: School of Computer Science and Engineering 
Rights: This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0).
Fulltext Permission: open
Fulltext Availability: With Fulltext
Appears in Collections:SCSE Theses

Files in This Item:
File Description SizeFormat 
thesis.pdfThesis4.82 MBAdobe PDFThumbnail
View/Open

Page view(s)

209
Updated on Jun 15, 2024

Download(s) 50

102
Updated on Jun 15, 2024

Google ScholarTM

Check

Altmetric


Plumx

Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.