OPTIMIZING SENTIMENT ANALYSIS WITH BERT ENHANCED BY BILSTM AND BIGRU LAYERS

As mobile technology develops rapidly, social media becomes a rich source of platform for people to voice for his or her opinions and point of views. In order to support business decision making and policy making, we need to quantify public moods, sentiments by conducting sentiment analysis which emerges as a key research area in recent year. Many research efforts have been put into performing sentiment analysis, many tools and algorithms are devised to determine the sentiment as positive, negative or neutral, in social media. We improve the sentiment classfier by training a classical machine learning model on three different data sets. Although BERT has shown good performance in sentiment analysis, we need to find a way to raise the accuracy. We propose four Deep Learning models by combining BERT with Bidirectional Long ShortTerm Memory (BiLSTM) and Bidirectional Gated Recurrent Unit (BiGRU) algorithms. Our paper uses pre, trained word embedding vectors as inputs and aims to increase the accuracy of sentiment analysis and test the effect of BERT‘s combination with BiGRU and BiLSTM layers, DistilBERT and RoBERTa. We test the classification of text sentiments with and without use of emoji symbols. A comparative analysis of 2 pre, trained BERT models and 7 classical machine learning models suggests that the models with BiGRU layers achieve best performance in our sentiment analysis pipelines