A Hybrid Topic Model Integrating SciBERT and LDA for Enhanced Topic Modeling

Enhancing topic modeling and document clustering is crucial to deepen the understanding of technology trends and identify emerging technologies. To fill the research gap of hybrid document clustering framework in scientific articles, this study proposes a novel hybrid model, Sci-LDAtopic, which integrates the strengths of a scientific domain-specific pretrained language model (SciBERT) and Latent Dirichlet Allocation (LDA) to address the dual challenges of insufficient semantic sensitivity in traditional topic models and weak interpretability in deep learning models. Specifically, the model concatenates contextual semantic embeddings generated by SciBERT with topic probability distributions extracted by LDA to form joint semantic representations, followed by clustering using the K-Means algorithm. Experiments on a dataset of 20,140 academic papers spanning computer science, physics, mathematics, and related fields demonstrate that Sci-LDAtopic significantly outperforms single-model approaches across key metrics: topic diversity (TD: 0.925, improvements of 12.1% and 23.3% over SciBERT and LDA), semantic coherence (C<inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">v</inf>: 0.563, gains of 13.5% and 8.9% compared to SciBERT and LDA), and clustering consistency (marginal optimizations in both Adjusted Rand Index and V-measure). The results confirm that synergistically integrating statistical topic distributions with deep semantic representations effectively captures both local topic structures and global semantic relationships in scientific literature. This approach provides a robust and interpretable solution for cross-disciplinary knowledge discovery.