Text-based Multi-pair Contrastive Learning Bidirectional Transformer for Sequential Recommendation ^*

Sequential recommendation aims to capture the dynamic changes in user interests by leveraging their historical behavior sequences for making recommendations. Existing methods often rely on explicit item IDs or generic textual features, but they struggle with cold start scenarios and adapting to new datasets. In this paper, we propose a novel approach called Text-based Multi-pair Contrastive Learning Bidirectional Transformer for Sequential Recommendation (TMCBiT). This method takes sequences of users' historical interactions in the form of key-value text pairs as input, enhancing representation capability through multi-layer embeddings. It employs a bidirectional Transformer model with a long-range attention mechanism, integrating random masking training with multi-pair contrastive learning for joint optimization. Experimental results demonstrate that our proposed method effectively alleviates issues related to cold starts and new dataset adaptation, significantly improving recommendation performance. This study provides a viable path for enhancing model performance in recommendation systems using deep learning techniques, offering valuable guidance for developing more intelligent, accurate, and personalized recommendation systems.