Hierarchical Sequence ID Representation of Large Language Models for Large-scale Recommendation Systems

Recently, large language models (LLMs) have shown great potential in recommendation systems (RS), however, there are still challenges in applying LLMs to large-scale RS. One major challenge is to effectively transform unique identities (IDs) in RS into a suitable training space for LLMs. Current approaches either lack collaborative signals of user behaviors, or they represent IDs with a single token, which fail to align with the autoregressive paradigm of LLMs. Consequently, the potential of IDs is not fully utilized in LLMs for recommendation, particularly in large-scale systems with numerous users, items, and sparse interactions. On the other hand, when enhancing large-scale RS, the high-dimensional representations from the last hidden layer of LLMs pose difficulties for meeting the low-latency requirements. In this paper, we propose a Hierarchical sequence ID representation method for applying LLMs to Recommender systems, called HI-RecM. Specifically, we design a tree-based ID encoding method that generates hierarchical sequence indices, then we embed the indices token-by-token into LLMs to align LLMs' autoregressive training paradigm. We design a joint dimension reduction (JDR) module to efficiently extract information, enabling lossless dimension reduction of high-dimensional LLM representations. Extensive experiments show that our method can outperform state-of-the-art methods. Moreover, HI-Rec has been deployed on Kuaishou's online advertising platform, achieving 2.1% increase of revenue. We also release the implementation code in https://github.com/Rita-73/HI-Rec.