Evaluation of Topic Independence in LDA and ITA

In recent years, the rapid expansion of the Internet and personal computing has led to the digitization of vast amounts of document data, resulting in a significant increase in the volume of textual information. Despite this growth, much of the data remains underutilized. To address this issue, research in text mining has focused on extracting useful knowledge from large-scale datasets, with topic extraction emerging as a major challenge. To tackle this challenge, topic models such as Probabilistic Latent Semantic Analysis (PLSA) and Latent Dirichlet Allocation (LDA) have been developed, both based on probabilistic generative frameworks. In contrast, Independent Topic Analysis (ITA) emphasizes the relationships between topics by employing independent component analysis to extract topics while maximizing their independence. Extracting highly independent topics offers several advantages, such as enabling the creation of summaries rich in diverse information and enhancing visualization techniques to improve user comprehension. Previous research on topic extraction has aimed to maximize topic independence. Although there have been evaluations of whether PLSA preserves independence among extracted topics, similar investigations for LDA are lacking. In this paper, we compare the mutual information between topics extracted by LDA and ITA to assess whether LDA maintains topic independence. Experimental results demonstrate that ITA consistently produces lower mutual information across various document datasets. This finding suggests that LDA tends to distribute highly important words across multiple topics, thereby reducing their independence. Future work will focus on mathematically analyzing the extent to which LDA preserves topic independence.