Leveraging RAG With Transformer for Context-Based Personalized Recommendations

Recent advancements in large language models (LLMs) have shown significant progress in addressing challenges related to data sparsity and the cold-start problem. In e-commerce, recommendation systems are widely used as strategic tools to boost sales and enhance the customer experience by helping users find relevant products. Custom LLMs, leveraging textual features from user feedback, have been successfully applied to recommendation systems, yielding improvements across various recommendation scenarios. However, most existing methods rely on training-free recommendation approaches, which depend heavily on pre-trained knowledge. When LLMs are trained on sparse data or lack historical information, their performance in recommendation systems can be negatively impacted. Furthermore, inference with LLMs tends to be slow due to autoregressive generation, which limits the efficiency of traditional recommendation methods. To address these challenges, our contributions are: We proposed the Retrieval Augmented Generation with Transformer Recommendation (RAGX11Rec) framework. This framework integrated LLMs with a transformer-based model in a two-step process: (i) RankRAG is used to filter the top-k preferences via tuning the LLM for effective context ranking, (ii) a transformer model with 11 embedded layers generated the top-N recommendations based on ranked preferences. Our instruction-tuned transformer module demonstrates superior performance by incorporating a fraction of ranked data into the training process. We evaluated the effectiveness of RAGX11Rec against state-of-the-art baseline methods using two public datasets taken from AliExpress and Epinions. Experimental results indicate that RAGX11Rec consistently outperforms other methods in recommendation accuracy and efficiency. Our key findings are; (i) RAGX11Rec effectively addresses the cold-start problem by leveraging retrieval-augmented generation (RAG) and transformer-based ranking. Unlike traditional models, it can deliver high-quality recommendations even when user interaction data is limited. (ii) Tested on public datasets, RAGX11Rec delivered consistent improvements over other models and proved its scalability and adaptability across diverse product categories. This suggests the framework is robust and adaptable enough for large-scale commercial use.