TSM-UniT: Two-Stage Model with Unidirectional Transformers for Sequential Recommendation

Current models for Sequential Recommendation begin to consider auxiliary information (e.g. attribute information). However, integrating attribute information and item information directly will destroy item embedding space. In addition, by theoretical analysis and experiments, we choose Unidirectional Transformers as base model. However, the loss function in Unidirectional Transformers only uses ground-truth item and randomly samples negative item, which doesn't make full use of the whole item set. To solve these problems, we propose a Two-Stage Model with Unidirectional Transformers for Sequential Recommendation (TSM-UniT). An attribute prediction stage is placed before the item prediction stage. In attribute prediction stage, attribute information is regarded as prediction target. Then, by passing parameters between these two stages, the item prediction stage can fully integrate the attribute information without destroying the embedding space. What's more, we use cross-entropy loss to replace binary cross-entropy loss in item prediction stage. Experiments on three public datasets show that our model boosts performance.