Optimal architecture for a sentiment analysis transformer with multihead attention and genetic crossover

Sentiment analysis, a key component of natural language processing, is of paramount importance in various fields such as strategic surveillance, online image management, and customer satisfaction assessment. However, despite recent advances, improving the accuracy and adaptability of models remains a crucial challenge. This paper presents a cutting-edge method that combines the Transformer with evolutionary optimization techniques to improve sentiment analysis results. We rely on the use of pre-trained models to obtain embeddings and language features, thus guaranteeing a rich and relevant textual representation. Our method, called OAST-MAGC (Optimal Architecture for a Sentiment Analysis Transformer with Multihead Attention and Genetic Crossover), is distinguished by incorporating a multihead attention mechanism optimized using a genetic crossover technique. The originality of our approach lies in the integration of dynamic pruning, which selects the most relevant attention heads before starting genetic optimization. This step reduces the search space, improves convergence speed, and produces a lighter and more efficient final model. Additionally, the weights of the final layers are combined through genetic crossover, which promotes more efficient generalization and reduces the risk of overfitting. This architecture, which merges the effectiveness of pre-trained models with a dynamic optimization approach, offers a significant improvement in the accuracy and efficiency of sentiment analysis tasks. Experiments have proven that our method outperforms current techniques in terms of performance and robustness, achieving an accuracy of 95.96% and an F1-score of 96%, opening the way to new possibilities in processing complex textual data.