Exploring the Efficacy of Large Language Models in Topic Modelling: A Comparative Analysis

Topic modelling is an important technique in natural language processing, used to discover topics or themes present within a given corpus. This paper investigates the performance of advanced Large Language Models (LLMs) at the task of topic modelling. We have evaluated the performance of LLMs such as BERT, RoBERTa, ALBERT and ELECTRA. Presently, the contribution discusses a paradigm shift from the classic topic modelling using models such as Latent Dirichlet Allocation (LDA) to the state-of-the-art approaches which adopt pre-trained models with the incorporation of contextual embeddings. With such embeddings and advanced tokenization schemes, the work investigates the capability of the models to catch latent semantic structures effectively. Experiments were conducted on the New York Times annotated corpus, where spherical K-means clustering algorithm was applied to project topics into common latent space. We used Normalised Mutual Information Score as our principal metric for evaluation, with BERT language model performing the best, giving a score of 0.45 among other language models; namely ALBERT, RoBERTa and ELECTRA. The results show two very different phenomena: BERT and ALBERT are substantially better in coherence and topic interpretability, while RoBERTa and ELECTRA showed limitations to keep topic coherence. These findings indicate how topic modelling could be improved, with the support of LLMs, to better balance coherence and diversity.