Sentiment analysis and monitoring for chatbot in education context

Abstract

Sentiment analysis and intent classification are essential to improve chatbot performance by allowing conversational systems to better understand user emotions and intentions.Sentiment analysis identifies and categorises opinions as positive, negative, or neutral, while intent classification deciphers the underlying purpose behind user inputs. Together, they allow chatbots to respond more empathetically and contextually, leading to more personalized and effective interactions. This project focuses on developing and evaluating models for both tasks to improve chatbot monitoring, decision-making, and response quality. The SC1015 Ask Narelle dataset is utilised to train and test our models, conducting both document-level and sentence-level classification across three sentiment categories and multiple user intents. A variety of machine learning architectures are explored. Additionally, pre-existing sentiment classifiers are employed, leveraging transfer learning to assess their efficiency and effectiveness. After comprehensive evaluation, the best-performing models are fine-tuned and deployed as real-time plug-in modules for chatbot developers to easily integrate into chatbots for analysis and response generation. To support continuous improvement and monitoring, an interactive dashboard is developed geared for chatbot developers. This provides comprehensive insights into chatbot performance, featuring time series visualisations, distribution charts, and summary statistics. Developers can apply customisable filters to focus on specific segments. The dashboard, combined with robust sentiment and intent analysis modules, enables developers to enhance user satisfaction by delivering more contextually-aware and emotionally intelligent interactions while ensuring continuous performance monitoring.