Topic modelling and sentiment analysis for public opinion mining of the #BBNaija reality TV show: a critical analysis

Abstract This study explores user engagement and sentiment during the Big Brother Naija (BBNaija) reality TV show using advanced natural language processing and deep learning techniques. There is a dearth of comprehensive studies employing advanced topic modelling techniques to dissect the multifaceted conversations surrounding the show. The research methodology integrated topic modelling, sentiment analysis, and deep learning models to provide a comprehensive understanding of user interactions on X (formerly Twitter) based on 110,651 tweets from four seasons of the show between 2020 and 2023. Various topic modelling techniques, including LDA, NMF, BTM, LSA, and GSDMM, were employed and evaluated with metrics such as Jaccard similarity, perplexity, and coherence scores. We constructed a 5% human-labelled gold standard (N = 5256) and reported inter-rater reliability (Krippendorff’s α, nominal = 0.606). Sentiment analysis was conducted using VADER and RoBERTa models. The study further involved a comparative analysis of deep learning models, including CNN, LSTM, Convolutional LSTM, and Convolutional Bidirectional Recurrent Neural Network, leveraging embeddings such as Word2Vec, FastText, USE, BERT, and Instructor Embedding to determine the most effective approach for sentiment prediction. A CNN model with BERT + Instructor embeddings emerges as the top performer, achieving 97% in recall, precision, F1-score, and accuracy. Conversely, the worst-performing combinations are Instructor Embedding with LSTM and Word2Vec + USE + FastText with LSTM, achieving much lower metrics. This study presents a rigorously evaluated framework that combines topic modelling, sentiment analysis, and deep learning to extract insights from large social media datasets. We evaluated the models against a human-labelled gold standard using fivefold cross-validation and statistical testing. Future work may extend the analysis to cover multiple social media platforms and multimodal contents.