Cross-Attentive Multi-Modal Topic Modeling for Richer Contextual Analysis of Text and Auxiliary Data
Topic modeling has traditionally relied on textual data to uncover hidden themes within a corpus. However, with the increasing prevalence of multi-modal content in research and applications, there is a need to extend topic modeling to incorporate auxiliary modalities such as images, audio, and metadata. The paper introduces Multi-Modal Generative Topic Modeling with Cross-Attentive Networks (MM-GTM-CAN), a framework that uses a cross-attentive neural network to fuse embeddings from multiple modalities. Textual data is processed using a transformer-based encoder, while auxiliary data is encoded using modality-specific architectures. The fused embeddings are then passed through a Conditional Variational Autoencoder (CVAE) to generate contextually rich topics. We assessed topic models on a dataset of 6743 BMC Bioinformatics articles (2005–2015), in an 80:20 split, using a shared vocabulary and three random seeds. Latent Dirichlet Allocation (LDA) with five topics obtained a C_V coherence score of 0.560 ± 0.028, an NPMI score of 0.065 ± 0.013, a topic diversity score of 0.953 ± 0.023, and a held-out perplexity score of 351.7 ± 3.0. LDA with twenty topics achieved stronger coherence (C_V: 0.663) but lower diversity (0.847). Thus, the number of subjects is a trade-off between coherence and repetition. Overall, the suggested system increases topic coherence and allows for broader contextual analysis than prior modal techniques.
Authors
- Mahalakshmi Guruvayur Suryanarayanan (ORCID: https://orcid.org/0000-0002-6286-2058)
- Sendhilkumar Selvaradjou (ORCID: https://orcid.org/0000-0001-6006-1866)
- Hemadharsana Sakkaraja (ORCID: https://orcid.org/0000-0003-3575-0138)
Institutions
- Anna University, Chennai (IN)
Publication Details
- Journal
- Applied Sciences
- Published
- 2026-11-20
- DOI
- https://doi.org/10.3390/app162211501
- Primary Topic
- Topic Modeling
- Type
- article
- Field-Weighted Citation Impact
- 0.00