Multimodal Sentiment Analysis of Online Memes: Integrating Text and Image Features for Enhanced Classification

Memes became a natural entity to express our thoughts and emotions in social media platforms like Instagram, Facebook, and Twitter. The memes sentiment analysis remains unexplored, mainly in academic research. We used the Bi-modal framework in this study to classify meme sentiments, which contains both the textual and visual elements. Our findings emphasize that analyzing text and images separately does not accurately capture the emotions conveyed by memes. The model should consider both text and images together to correctly extract the mixed feelings contained within a meme. Consequently, our Bi-modal model framework harmonizes textual and visual attributes to augment classification accuracy. Additionally, we endorse the application of an attention mechanism to refine meme classification precision further. The results generated by our framework substantiate its effectiveness, yielding impressive accuracy rates and macro F1 and micro F1 scores of 59.48%, 39.04%, and 60%, respectively. The results show the potential of our Bi-modal model in meme sentiment analysis allowing for the better understanding of an emotions and sentiments that conveyed through the memes inonline communication.