Capsule-enhanced RoBERTa for hierarchical sentiment analysis on social media texts

Sentiment analysis plays a vital role in natural language processing by extracting subjective information from user-generated content, especially on social media platforms. Despite significant progress with transformer-based models, existing approaches struggle with accurately identifying neutral sentiments, handling inconsistent token lengths, and capturing hierarchical relationships within text. Traditional RNN-based hybrids add sequential modelling capacity but often suffer from inefficiency and limited interpretability. This study proposes a novel hybrid architecture integrating RoBERTa with Capsule Networks, aiming to enhance both contextual embedding and hierarchical feature modelling. Capsule routing enables the model to preserve spatial and part–whole sentiment relationships that are typically overlooked in sequential models. Three benchmark datasets—Twitter US Airline Sentiment, Apple Twitter Sentiment (CrowdFlower), and Apple Twitter Sentiment Texts—were used to evaluate performance across diverse domains with noisy, emoji-rich, and short-text content. Experimental results demonstrate that the RoBERTa + CapsNet model consistently outperforms RoBERTa-only and RoBERTa-RNN hybrids. It achieves up to 91.72% accuracy and notable improvements in F1-score, Cohen’s Kappa, and AUC, particularly for the challenging neutral class. Furthermore, the model reduces training time and latency jitter compared to sequential hybrids, highlighting its suitability for real-time applications. Overall, this research establishes Capsule-Transformer integration as a promising direction for sentiment analysis. The proposed framework not only advances performance and interpretability but also lays the groundwork for future studies on capsule-enhanced architectures in natural language processing.