Leveraging LLMs for Integrated Sentiment and Topic Analysis on African Social Media

Leveraging AI to analyze key topics on African social media can enhance public governance. Our study analyzes social media discourse within African society on development concerns by (1) evaluating AI techniques for sentiment, topic, and theme extraction, comparing the accuracy of these methods with human annotations, and (2) extracting key insights from the data to provide policymakers with actionable recommendations for sustainable development. For this study, we utilized a data corpus of 22,036 posts from Twitter and YouTube, all focused on development issues in Africa. We applied topic modeling to extract relevant topics from the corpus and used similarity analysis, powered by Large Language Models, to link these topics to prevalent development themes. Additionally, we leveraged unsupervised models such as VADER and Large Language Models to extract sentiment related to the identified topics. To validate these model-generated sentiments, we conducted a small crowdsourced study to gather human-annotated labels as ground truth. Our sentiment analysis findings show improvements with models like TextBlob, VADER, and Llama. Fine-tuning, partic-ularly with BERT, achieved an impressive Fl score of 0.988. Meanwhile, Llama demonstrated strong precision (0.72) and balanced accuracy (0.55) in capturing contextual sentiment. We identified 304 topics using BERTopic and Llama, with robust coherence (0.81 C-v) and divergence (0.58 IRBO). In theme analysis, the One-vs-Rest classification with ensemble voting performed exceptionally well, with ‘Poverty’ achieving the highest F1 score of 0.89. Our results suggest that African policymakers prioritize addressing corruption, unemployment, drought, and instability, while closely monitoring the positive impacts of policy interventions.