Cross-Attentive Multi-Modal Topic Modeling for Richer Contextual Analysis of Text and Auxiliary Data

Topic modeling has traditionally relied on textual data to uncover hidden themes within a corpus. However, with the increasing prevalence of multi-modal content in research and applications, there is a need to extend topic modeling to incorporate auxiliary modalities such as images, audio, and metadata. The paper introduces Multi-Modal Generative Topic Modeling with Cross-Attentive Networks (MM-GTM-CAN), a framework that uses a cross-attentive neural network to fuse embeddings from multiple modalities. Textual data is processed using a transformer-based encoder, while auxiliary data is encoded using modality-specific architectures. The fused embeddings are then passed through a Conditional Variational Autoencoder (CVAE) to generate contextually rich topics. We assessed topic models on a dataset of 6743 BMC Bioinformatics articles (2005–2015), in an 80:20 split, using a shared vocabulary and three random seeds. Latent Dirichlet Allocation (LDA) with five topics obtained a C_V coherence score of 0.560 ± 0.028, an NPMI score of 0.065 ± 0.013, a topic diversity score of 0.953 ± 0.023, and a held-out perplexity score of 351.7 ± 3.0. LDA with twenty topics achieved stronger coherence (C_V: 0.663) but lower diversity (0.847). Thus, the number of subjects is a trade-off between coherence and repetition. Overall, the suggested system increases topic coherence and allows for broader contextual analysis than prior modal techniques.

Authors

Institutions

Publication Details

Journal
Applied Sciences
Published
2026-11-20
DOI
https://doi.org/10.3390/app162211501
Primary Topic
Topic Modeling
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Cross-Attentive Multi-Modal Topic Modeling for Richer Contextual Analysis of Text and Auxiliary Data

Mahalakshmi Guruvayur Suryanarayanan, Sendhilkumar Selvaradjou, Hemadharsana Sakkaraja
1 links
Applied Sciences
Topic Modeling
article

Cross-Attentive Multi-Modal Topic Modeling for Richer Contextual Analysis of Text and Auxiliary Data

Mahalakshmi Guruvayur Suryanarayanan, Sendhilkumar Selvaradjou, Hemadharsana Sakkaraja
article en

Abstract

Topic modeling has traditionally relied on textual data to uncover hidden themes within a corpus. However, with the increasing prevalence of multi-modal content in research and applications, there is a need to extend topic modeling to incorporate auxiliary modalities such as images, audio, and metadata. The paper introduces Multi-Modal Generative Topic Modeling with Cross-Attentive Networks (MM-GTM-CAN), a framework that uses a cross-attentive neural network to fuse embeddings from multiple modalities. Textual data is processed using a transformer-based encoder, while auxiliary data is encoded using modality-specific architectures. The fused embeddings are then passed through a Conditional Variational Autoencoder (CVAE) to generate contextually rich topics. We assessed topic models on a dataset of 6743 BMC Bioinformatics articles (2005–2015), in an 80:20 split, using a shared vocabulary and three random seeds. Latent Dirichlet Allocation (LDA) with five topics obtained a C_V coherence score of 0.560 ± 0.028, an NPMI score of 0.065 ± 0.013, a topic diversity score of 0.953 ± 0.023, and a held-out perplexity score of 351.7 ± 3.0. LDA with twenty topics achieved stronger coherence (C_V: 0.663) but lower diversity (0.847). Thus, the number of subjects is a trade-off between coherence and repetition. Overall, the suggested system increases topic coherence and allows for broader contextual analysis than prior modal techniques.

Applied SciencesVol. 16(22)
Anna University, Chennai (IN)
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon