F2MSA2: Integrating Mamba and Self-Attention for Sequential Recommendation

Sequential Recommendation (SR) has been widely used in various Internet platforms, such as content streaming platforms and e-commerce. While the Self-Attention (SA) mechanism has been shown to be effective in capturing global dependencies, its quadratic complexity makes it less efficient. Mamba can improve efficiency, but its unidirectional architecture makes it difficult to capture the full context of a sequence, preventing further development.To address these issues, this work proposes a sequential recommendation model called F2MSA2 This model combines Mamba and SA for hybrid modelling. A modified version of the Mamba block, flipped Mamba (FMA), is first introduced, which flips the interaction sequences and then serves as an input to the Mamba block to enhance the model's ability to acquire and utilize future information. Secondly frequency-domain SA (FSA) transforms the SA from the time domain to the frequency domain in order to acquire both low-frequency and high-frequency information, where the low-frequency information usually reflects the user's long-term stable preferences while the high-frequency information reveals the user's short-term fast-changing interests. And several frequency-domain feed-forward networks are constructed to learn external factors and mitigate their effects on user preferences.