Knowledge-Enhanced Multi-Level Session Graph Model for Interactive Recommendation through Deep Reinforcement Learning

Deep reinforcement learning (DRL), has shown promise in solving intractable challenges in interactive recommendation systems (IRS). In DRL-based interactive recommendation, state modeling is vital for well-capturing users’ continuous interaction behaviors with recommendation systems. To effectively capture the behavior of users, existing works for state modeling have evolved from sequential-based modeling to session-based modeling. However, existing session-based state modeling works in IRS are still not fully explored with premature session models and insufficient fusion for different session features. As a result, they cannot capture complicated session patterns during interaction, leading to significant information loss. In this paper, we propose a Knowledge-enhanced Multi-Level Session Graph (KMSG) model for interactive recommendation to address the above challenge. KMSG models the user interactive data into multi-level session graphs and effectively encodes the states via graph neural networks. Specifically, a novel 3-level item transition graph is designed to capture the common session patterns and intra-session item transitions. We further utilize the information from knowledge graph to enhance the item relations in KMSG. We then design an attention-based graph neural network to propagate the information in KMSG. Extensive experiments on four real-world benchmark datasets demonstrate the superiority of KMSG over state-of-the-art baselines and the rationality of our design in KMSG.