Enhanced sentiment analysis on social media using BERT and multimodal attention-based fusion

Sentiment analysis on social media is increasingly challenging due to the growing prevalence of multimedia posts to convey complex emotions. Traditional sentiment analysis methods, which rely solely on textual or visual information, often fail to capture the nuanced interplay between modalities, leading to suboptimal predictions. Motivated by the limitations of unimodal approaches in handling sarcasm, this study proposes a novel multimodal sentiment analysis model that integrates textual and visual features for enhanced understanding. The proposed model leverages BERT and BiLSTM networks for capturing contextual and sequential dependencies in text, while CNNs extract rich spatial features from images. An attention-based fusion mechanism further enhances the integration of these modalities by focusing on the most informative elements in both text and pictures. Pretrained CNNs are fine-tuned through transfer learning to improve visual feature extraction, and the combined multimodal representation is used for sentiment prediction. Experimental results demonstrate that the proposed model achieves 92% accuracy, 89.5% precision, 89% recall, and an F1-score of 89.2%, outperforming conventional unimodal and existing multimodal approaches. This work lays the foundation for future extensions to include other modalities such as audio or video, enabling more comprehensive emotion understanding in real-world applications.