EGBERT: A EGYPTIAN DIALECT PRE-TRAINED LANGUAGE MODEL

The Arabic language encounters difficulties in Natural Language Processing owing to its intricate morphology and scarce resources. Transformers like BERT have proven effective in language comprehension, but most models are trained in widely spoken languages like English, French, and German. With the recent rise of social media platforms has rendered sentiment analysis a crucial instrument in natural language processing, facilitating the identification of individuals' emotions, opinions, perceptions, and responses to major events. This research introduces an Egyptian model, EgBERT, employing the Masked Language Modeling pre-training technique. The model produces a word representation for the Egyptian Arabic dialect with more than one million words sourced from social media platforms. Through model fine-tuning, it attains enhanced results relative to prior BERT-based models and establishes a new benchmark in sentiment analysis tasks. The findings suggest that pre-training a specialized model on a constrained dataset can surpass the efficacy of existing models learned on larger datasets. Among the pre-trained models used in this study, the EgBERT model achieved the highest accuracy rate of 62.96% according to the test dataset findings