Relevancy classification in Bangla social media comments using Transformer architecture

Comments on social media posts reflect public sentiments and provide valuable feedback, contributing additional insight to the discussion. However, with the expansion of social media platforms, irrelevant comments have become a source of disturbance for users, causing confusion and impeding meaningful conversations. As social media usage grows exponentially, manual filtration of irrelevant comments has become unfeasible, necessitating the development of an automated framework to address this issue. Although previous studies have been conducted to automate this process in the English language, this domain remains largely unexplored for low-resource languages, such as Bangla. To date, there is no publicly available labeled dataset specifically designed for relevancy classification in the Bangla language, creating a gap in applying existing models and frameworks to Bangla text. Most studies on the Bangla language have focused on tasks such as hate speech, aggression, and spam detection, with no attention given to the development of systems for identifying irrelevant content. To address this gap, we developed a novel dataset comprising 16,970 Bangla comments from 176 Facebook posts. Baseline experiments were conducted on the dataset using three transformer models: BanglaBERT, DistilBERT, and XLM-RoBERTa, where XLM-RoBERTa achieved the highest accuracy of 75.49% and an F1-score of 75.49%.