Enhancing neural topic modeling for social media text via semantic bag of word clusters and log-domain Sinkhorn transport

Topic modeling has been widely applied to analyze text data from social media platforms. Under this scenario, traditional Neural Topic Models (NTMs) encounter three primary challenges: (1) initial text representation; (2) the long-tail nature of topic distributions in social network texts; (3) approximation of Optimal Transport. Motivated by these challenges, we propose an end-to-end solution spanning from text representation to topic modeling. First, we propose SBoWC, a novel text representation method that performs dimensionality reduction while absorbing semantic information through base terms, achieved by combining word embeddings with clustering statistics. Subsequently, we propose GSWTM, a Wasserstein-based autoencoder topic model that fits the long-tail topic distribution in social network texts via Gamma priors and innovatively employs log-domain Sinkhorn to approximate Optimal Transport. Ablation studies demonstrate the transferability and effectiveness of SBoWC in text representation. GSWTM demonstrates significantly better performance than baselines in TU, C V , and the comprehensive metrics TQ across four real social network datasets of varying sizes. The log-domain Sinkhorn approximation exhibits excellent stability, allowing the regularization parameter ϵ to be reduced to 0.1–0.01, thereby approaching the original Optimal Transport.