Augmenting large language models for financial sentiment analysis: a heuristic sparse mixture-of-experts framework

Financial sentiment analysis (FSA) is essential for extracting actionable market insights from unstructured textual data to support investment decisions and risk management. Although large language models (LLMs) demonstrate remarkable capabilities in general natural language processing, they nevertheless face challenges in comprehensively capture the domain-specific semantics, numerical relationships, event impacts, and implicit sentiment expressions critical for robust financial sentiment interpretation within a unified framework. To address these limitations, this study proposes Heuristic Sparse Mixture-of-Experts framework for Financial Sentiment Analysis (HSMoE-FSA), a novel framework designed to augment LLMs for financial sentiment analysis. Specifically, the framework integrates four specialized LLM-based experts: Financial Semantic Comprehension (FinSem) for interpreting financial terminology, Financial Numeric Reasoning (FinNum) for modeling quantitative sensitivities, Financial Event Impact Assessment (FinEvent) for evaluating market-impact events, and Financial Implicit Sentiment Decoding (FinSent) for detecting latent sentiment cues. Leveraging an entropy-based routing mechanism, the framework dynamically activates optimal expert combinations through Top-K sparse gating and weighted aggregation, thereby enabling synergistic multi-dimensional feature fusion. Furthermore, a phased training strategy decouples expert module learning from router calibration to ensure stability. Extensive evaluations on benchmark datasets demonstrate that HSMoE-FSA achieves state-of-the-art performance, with weighted F1-scores ranging from 89.67% to 92.29%, representing absolute improvements of 0.72% to 1.74% over current leading methods. Moreover, qualitative analysis further confirms the framework’s superior capability in resolving nuanced financial semantics and implicit sentiment through expert collaboration.