Mixture-of-Experts Based LLM Model For Financial Text Classification

Automatic news topic classification is a fundamental task in natural language processing, facilitating efficient organization and retrieval of news articles. The classification of financial news is particularly challenging due to the complex language structures and domain-specific terminology. This study investigates the effectiveness of large language models, specifically the Mixture-of-Experts (MoE) model Qwen-MoE, in addressing these challenges. We propose a novel fine-tuning approach using QLora and a tailored instruction-tuned dataset. Experimental results on a Twitter financial news classification dataset demonstrate that our model outperforms existing state-of-the-art models.