Enhancing Multi-Turn Dialogue Generation with MoE-Based Multi-Latent Variable Fusion

Generating relevant and diverse responses remains challenging for non-pretrained small models in open-domain multi-turn dialogues. To address the one-to-many mapping problem caused by conversational uncertainty, we propose a novel framework integrating Variational Autoencoder (VAE) with a Mixture-of-Experts (MoE) model. The VAE component learns three distinct latent variables: background, roles, and topic, to capture dynamic conversational factors, while the MoE module employs context-aware gating to dynamically activate domain-specific experts based on these variables. Extensive experiments on multi-turn dialogue datasets show that our approach outperforms state-of-the-art non-pretrained baselines, particularly in improving the relevance and diversity of responses.