A real-time predicting online tool for detection of people’s emotions from Arabic tweets based on big data platforms

Abstract Emotion prediction is a subset of sentiment analysis that aims to extract emotions from text, speech, or images. The researchers posit that emotions determine human behavior, making the development of a method to recognize emotions automatically crucial for use during global crises, such as the COVID-19 pandemic. In this paper, a real-time system is developed that identifies and predicts emotions conveyed by users in Arabic tweets regarding COVID-19 into standard six emotions based on the big data platform, Apache Spark. The system consists of two main stages: (1) Developing an offline model and (2) Online emotion prediction pipeline. For the first stage, two different approaches: The deep Learning (DL) approach and the Transfer Learning-based (TL) approach to find the optimal classifier for identifying and predicting emotion. For DL, three classifiers are applied: Convolutional Neural Network (CNN), Gated Recurrent Unit (GRU), and Bidirectional GRU (BiGRU). For TL, five models are applied: AraBERT, ArabicBERT, ARBERT, MARBERT, and QARiB. For the second stage, create a Transmission Control Protocol (TCP) socket between Twitter’s API and Spark used to receive streaming tweets and Apache Spark to predict the label of tweets in real-time. The experimental results show that the QARiB model achieved the highest Jaccard accuracy (65.73%), multi-accuracy (78.71%), precision-micro (78.71%), recall-micro (78.71%), f-micro (78.71%), and f-macro (78.55%). The system is available as a web-based application that aims to provide a real-time visualization of people’s emotions during a crisis.