Momentum Contrastive Bidirectional Encoding with Self-Distillation for Sequential Recommendation

In this paper, we propose a new Momentum Contrastive Bidirectional Encoding network with S elf-D istillation (MoCoBE-SD) to alleviate the data sparsity and noise issues in sequential recommendation by providing rich informative supervisions from both sequence-level and item-level perspectives. In particular, a Momentum Contrastive Bidirectional Encoding (MoCoBE) network is first proposed by constructing momentum updated encoder based on an online bidirectional self-attention encoder, where a momentum contrastive learning task and a masked item prediction task are simultaneously optimized. Building upon MoCoBE, a well-elaborated Self-Distillation (SD) scheme is incorporated to further suppress the noise influence. Specifically, a well-trained sequence encoder by MoCoBE is adopted as the teacher encoder to provide refined supervision for the masked item prediction, which constitutes our MoCoBE-SD framework. Extensive experiments on three public datasets show that MoCoBE-SD outperforms the existing state-of-the-art methods consistently.