Sentiment Analysis Multi-Label of Toxic Comments using BERT-BiLSTM Methods

The number of individuals accessing the Internet rises steadily each year in Indonesia. One of the reasons behind this trend is the growing popularity of online platforms like Twitter, where people can freely share their thoughts and ideas. However, it is crucial to recognize that social media can occasionally serve as a breeding ground for negative behavior, such as the spread of toxic opinions. Additionally, more information will be collected as more people become active and share their views. Therefore, a technique capable of handling this textual data comprehensively, like classification, is needed. Simple categorization methods could be more effective since they split comments into positive and negative groups. Thus, to overcome such problems, multilevel classification assigns multiple labels to a single instance, allowing its categorization into different categories inside a single statement. This research uses a combination of BERT and BiLSTM methods, whereas BERT is used to obtain word vector values and will then be used as input in the BiLSTM model to perform multi-label classification tasks. This study uses two types of word vectors, the summation of the last four hidden layers and the final hidden layer of BERT, to create a better comparison model. The study achieved an accuracy of 0.889, precision of 0.925, recall of 0.917, and an F1 score of 0.91 for the model that used the last four hidden layers of BERT as word vectors