Automated essay scoring with incremental learning through crowdsourcing

Abstract

This research proposes a novel word-character fusion mechanism based deep learning model for Automated Essay Scoring (AES) with an Incremental Learning (IL-AES) approach through crowdsourcing. While many recently proposed deep learning models for the AES task utilize only word level information, the proposed word-character fusion mechanism incorporates both word level and character level information for essay scoring. Experiments are performed on the proposed deep learning AES model to demonstrate that it achieves the state-of-the-art performance on the Automated Student Assessment Prize (ASAP) dataset from Kaggle. Crowdsourcing is becoming increasingly popular in recent years among tasks that require collecting data in a scalable and efficient manner. The proposed IL-AES approach aims to improve the scoring accuracy of the AES model by training it incrementally with data collected through the crowdsourcing environment. The effectiveness of different components of the IL-AES approach is demonstrated by ablation studies presented in the report. A web-based prototype system is implemented to demonstrate how the proposed deep learning AES model adopting the word-character fusion mechanism and the proposed Incremental Learning approach are applied in a crowdsourcing environment.