Optimization Evaluation for Enhancing Deep Learning Performance in Arabic Text Classification

Optimization is a crucial process in training neural networks. Its goal is to find a set of parameters that offer the best performance for a specific problem. With the advent of deep learning models, which are primarily based on neural networks, interesting results have been achieved in NLP tasks, specifically in text classification. Furthermore, optimizing the parameters of these models is crucial to improve performance and reduce training time. Several optimization algorithms have been developed, such as those based on adaptive technique, which have proven their effectiveness by providing superior results compared to other models. In this work, a Convolutional Neural Network (CNN) model was developed, composed of multiple layers. Additionally, two Arabic datasets were used in this study: the first comprising ten different categories of text extracted from news platforms, and the second being a sentiment analysis dataset containing Arabic tweets of two polarities, positive and negative. Several optimizers do exist nowadays, in this study, we choose seven popular deep learning optimizers, namely, AdamW, Adamax, Lion, Nadam, Adam, Adam-Weight-Decay and RMSProp. The experimental outcomes demonstrate that the Adam and AdamW optimization algorithms delivered superior performance in comparison to other optimization techniques when applied to the task of topic classification. These two optimizers have proven to be highly effective, producing results that stand out among their counterparts. On the other hand, when it comes to sentiment analysis, the Nadam optimizer has shown to surpass the performance of the other optimizers. This indicates that the choice of optimizer can improve the performance of the results, and the optimal choice may vary depending on the task.