A Comparative Exploration of BERT, RNN, and GRU for Sentiment Classification

Sentiment classification tasks are a large branch of current natural language processing problems. The purpose of this task lies in determining the sentiment that the text is trying to convey given a piece of text or sentence. This technique can be widely referenced in areas such as, monitoring market opinion, protecting social security, and developing ai customer service. Currently, the most advanced solutions to the sentiment classification problem are based on Bidirectional Encoder Representations from Transformers (BERT), in order to try to research a better solution, this study will start from the bottom, first find the optimal Recurrent Neural Network (RNN) model to pave the way for future improvements. The research method is to start from one layer and keep adjusting the parameters or RNN types until the best model is found. Through an experimental study, it was found that the two-layer bidirectional Long Short-Term Memory (LSTM) performed best on the sentiment classification problem with an accuracy of 93.21%. It proves that LSTM is better than Gated Recurrent Unit (GRU) and simple RNN in understanding long text, and in the future, more datasets can be collected from other platforms, not twitter, and then the model can be trained more complexly, which will help the model to be generalized to other domains or occasions.