Sentiment detection with bidirectional encoder representations from transformers

Abstract

Sentiment Recognition is one of the NLP (Natural Language Processing) techniques to extract the information from the textual data. By extracting the sentiment polarity from the text of feedbacks and reviews, the organization can easily gain insight into the public opinions on their product or service. By doing so, the organization can use it to understand their brand reputation and fulfill the customer's need. Therefore, it is important to build an accurate model to detect the sentiment from the text. But the human language is very compound and includes much uncertainty such as sarcasm, polysemy which will harder for the machine to detect and analyze. Hence, it is a difficult task to detect the sentiment from the text. In this project, we will use the state-of-the-art deep-learning approach to detect the sentiment for the binary polarity (i.e. positive or negative) of the reviews/ opinions. We will use the Transformer Models block with the Bidirectional Encoder Representations from Transformers (BERT). This project will study the use of the BERT with and without additional fine-tuning and then propose the best method with high accuracy and high efficiency. Additionally, we will also further explore pre-training the BERT from scratch and compare against with the available pre-trained model. At the final step, we will implement a mobile application to allow the user to predict the sentiment from the text.