A Comparative Analysis of Transformer Architectures for Sentiment and Emotion Classification

This paper is a comparison of four Transformer model (BERT, ALBERT, T5, and XLNet) in the case of sentiment analysis of tasks based on data from social media. Used were two data sets: the X (Twitter) data set with tweets of messages related to games and the Emotion data set labeled in the anger, joy, and fear categories. Trained was the model in the same settings of preprocessing, training, as well as in the test settings for 3, 5, 7, and 10 epochs. Presented were results that indicated that the accuracy increased with the higher number of epochs. The maximum accuracies occurred in the case of the BERT model—88.63% in the case of the X data set as well as in the case of the Emotion data set, 97.05%. XLNet established great potential for long-range dependencies, and ALBERT obtained balanced performance due to lightweight architecture. On the contrary, performance of T5 was less in comparison to others. Generally, it could be inferred that Transformer architecture is superior to traditional machine learning technique in the context of sentiment analysis due to higher accuracy and better contextual understanding.