Identification of the severity level of a hate text by YouTubers

Hate speech has grown to be a serious problem on social media, encouraging antisocial behaviour and platform abuse due to the ability to disseminate content to millions within seconds, often anonymously. This study contributes to the research community in two ways when addressing this problem. Firstly, a crowdsourcing platform is used to annotate YouTube data to assess the content’s severity level, which is used as the training dataset. Secondly, it uses XLM-RoBERTa, a transformer-based multilingual model designed for code-mixed and code-switched text that includes Sinhala, English, and Singlish, to automatically classify the severity level of hate speech. At the initial stage, YouTube text data was initially divided into hate and non-hate categories. Hate texts were then further divided into three severity levels: Bottom, Intermediate, and Top. The XLM-RoBERTa model was used to fine-tune a dataset of 15,000 YouTube titles and comments for multi-class classification. With an F1-score of 0.91 and an accuracy of 92%, the model outperformed deep learning techniques like LSTM and conventional models like Random Forest, Support Vector Machines, and Logistic Regression. This is regarded as the first experiment based on the “UN Strategy and Plan of Action on hate speech”, which offers a unique framework for automatically classifying hate speech according to severity. It gives governments, platform users, and policymakers useful resources to help them make sound decisions. Future researchers can utilize this technique to rank the influence of hate speech on hate users to counteract hate speech propagation at a more severe level.