Deep Learning Algorithms Exploration to Model Toxic Comment Classification

Cyberbullying is not confined to Indonesia; it is a pervasive global issue. According to a survey conducted by the Pew Research Center, 41 % of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{1 0, 0 9 3}$</tex> U.S. adults reported experiencing cyberbullying or online harassment. Advancements in technology, particularly in machine learning and deep learning, present promising opportunities for addressing the issue of cyberbullying. Automated detection systems, driven by sophisticated algorithms, can analyse online interactions and identify patterns associated with bullying or harassment. Building on this premise, the objective of this research is to identify the most suitable model for classifying toxic comments by developing and evaluating three distinct models: LSTM (Long Short-Term Memory), BiLSTM (Bidirectional Long Short-Term Memory), and BERT (Bidirectional Encoder Representations from Transformers). The results highlight two major insights. First, the model trained with BERT generally outperforms LSTM and Bi-LSTM approaches, especially when sampling is appropriately tuned, thanks to BERT's superior capacity for language representation. Second, sampling can have both benefits and drawbacks. It can help improve balance metrics, like with BERT with base and large pre-trained weights. However, it can also lead to overfitting if the model isn't well regularised, as seen with LSTM and Bi-LSTM models. This research also aims to support the UN's SDG goals, in particular SGD-4 (Provide Quality Education).