Machine learning methods for detecting depression in Arabic tweets: a comprehensive performance analysis with enhanced evaluation metrics

Abstract This study presents a comprehensive evaluation of machine learning (ML) methods for the early detection of depression in Arabic tweets, addressing critical gaps in mental health informatics for Arabic-speaking populations. We introduce the ArabMindGuard (AMG) dataset, which comprises 3083 expertly annotated tweets from Saudi Arabia, and perform systematic comparisons across five datasets that encompass multiple Arabic dialects, including Saudi, Jordanian, and Modern Standard Arabic (MSA). Our research employs a rigorous evaluation framework, incorporating multiple performance metrics such as precision, recall, F1-score, and statistical significance testing. Leveraging advanced Arabic-specific natural language processing (NLP) techniques–including Term Frequency-Inverse Document Frequency (TF-IDF), N-gram analysis, and CAMeL tools–we assess the performance of nine machine learning classifiers with statistical rigor. The results show that ensemble methods and Support Vector Machines (SVM) consistently outperform other approaches, achieving F1-scores ranging from 0.82 to 0.95 across the datasets. These findings highlight the significant role of dialectal variation in depression detection and provide foundational baseline metrics for future deep learning applications. Overall, this study makes a valuable contribution to culturally responsive digital mental health interventions in the Arab world and establishes methodological benchmarks for Arabic mental health text analysis.