VARIUM+: Semantic Expert-Structured Memory-Augmented Variational Autoencoder for Multi-Interest Recommendation

Variational autoencoder (VAE) frameworks for multi-interest discovery have demonstrated strong recommendation performance. However, because VAEs model users independently, they fail to capture shared interest patterns among like-minded users, a key limitation of existing approaches. The previous work, Varium [41], addressed this by introducing a flat structured memory with iterative refinement to capture reusable interest patterns across users. In this paper, we extend Varium into Varium+, a novel semantic expert-structured memory-augmented VAE that fundamentally rethinks how shared interests are organized and retrieved. Unlike its predecessor, Varium+ organizes memory as a mixture of semantically-grounded experts, each dedicated to a different modality (textual, visual, collaborative) and composed of multiple prototypes representing fine-grained interest groups. For each user, we first discover multiple intra-user interest factors behind their item adoptions. Intra-user interest factors next attend to these expert memories via cross-attention to retrieve relevant inter-user signals, then pass through self-attention to model interactions among retrieved interests. The resulting shared interest representations are integrated back into each user’s multi-interest profile, enriching personal preferences with collaborative wisdom. Experiments on real-world datasets demonstrate that Varium+ consistently outperforms both its predecessor and a comprehensive set of baselines, with particularly strong gains when leveraging semantic knowledge. Qualitative analysis further reveals that our expert-structured memory organizes the latent interest space into well-separated clusters, validating its ability to discover meaningful shared patterns among users.