Amd'SaEr: Arabic Multimodal Dataset for Sentiment Analysis and Emotion Recognition

Multimodal sentiment analysis and emotion recognition have attracted significant interest in multimodal learning. Naturally, humans express their feelings and emotions through nuanced expressions across various verbal and non-verbal modalities. Despite this, there remains a critical gap in publicly accessible multimodal datasets for the Arabic language. To address this issue, we posited that creating a large and high-quality Arabic multimodal dataset would significantly improve sentiment analysis and emotion recognition in Arabic contexts. We aimed to develop a large, high-quality Arabic Multimodal Sentiment Analysis and Emotion Recognition (A md ’S a E r ) dataset by building upon our AMSA dataset, increasing its size to 1,037 samples, and adding emotional labels. Leveraging a novel methodology, we carefully selected and annotated data across audio, text, and visual modalities, and proposed a hybrid inter-annotator agreement strategy. Extensive analyses were conducted to validate the robustness of the dataset. We experimented with the A md ’S a E r dataset using a customized MERBench framework, which demonstrated the dataset’s efficacy and reliability. Our findings indicate the high quality of the dataset and underscore the importance of multimodal context for accurate sentiment analysis and emotion recognition in Arabic. We recommend further research and application of the A md ’S a E r dataset in broader Arabic contexts, as it provides a valuable resource for advancing multimodal analysis in this language.