Mining Topics towards ChatGPT Using a Disentangled Contextualized-neural Topic Model

Mining topics relevant to the advanced AI dialogue system, such as ChatGPT, from short-length posts on social media poses several challenges for existing topic-mining approaches. Firstly, Bag-Of-Words approaches, including probabilistic topic models and their embedding-based variants, may struggle to extract interpretable topics due to insufficient word co-occurrence. Secondly, contextualized based approaches, built on the autoencoding framework, often yield entangled topic spaces, resulting in the mixing of irrelevant words into topics. To address these limitations, we propose a novel Dis entangled Contextualized-neural Topic Model (DisCTM) based on textual representation learning. DisCTM leverages a pre-trained transformer language model to incorporate word sequence information and deal with the sparsity in short text. Additionally, it employs a topic disentangling mechanism to decorrelate dimensions of the latent topic space, effectively separating semantically irrelevant words into different topics. Extensive experiments have been conducted on three publicly available text corpora, and the results demonstrate the effectiveness of DisCTM in extracting high-quality topics, as measured by topic coherence and diversity metrics.