A Multi-View Text Classification Framework Combining Transformers and Topic Modeling

Text classification models often rely on transformer-based embeddings or topic-modeling techniques independently, limiting their ability to capture both contextual and thematic information. We propose the Topic-Transformer (TT) framework, a multi-view learning framework that combines transformer-based encoders and topic-modeling methods to improve text classification performance. TT processes text simultaneously through a transformer model and a topic-modeling technique. The resulting contextual embeddings and topic representations are then fed into a classification head. TT is tested using five distinct transformer encoders and three topic-modeling techniques, resulting in fifteen configurations of TT. The tests are conducted on six benchmark datasets and the results are evaluated in terms of macro-average F1-score, precision, recall, and training time. The results show that, for every combination of dataset and transformer, at least one TT configuration outperforms its standalone transformer counterpart. However, the magnitude and statistical significance of these improvements vary across datasets and configurations. In four of the six datasets, at least one TT configuration yields a statistically significant improvement at a significance level of 0.05 after multiple-testing correction. The largest significant gains are observed on 20Newsgroups, where improvements reach 2.8% in macro-average F1-score, with statistically significant improvements found in most transformer–TT comparisons.