Hybrid transformer-based recommender system with LLM-assisted semantic modeling for sequential and federated learning

Personalized content delivery is a core part of intelligent platforms of the modern era, where recommender systems need to be capable of functioning on large, sparse, and dynamically changing user-item interactions. Even though Transformer-based sequential recommendation models have been proven to be highly competent in terms of capturing temporal behavioral dependencies, they tend to be poor in semantic learning and poor in cold-start and data-sparsity settings. In order to tackle these limitations, the hybrid recommendation framework (LLMTransRec) proposed in this paper is a modular Transformer-based framework that combines collaborative user-item interaction signals with content-based genre features and semantic embeddings based on large language models (LLMs). The model proposed will learn identity-based representations, metadata-based content representations, and LLM-enriched semantics of textual content together in one self-attention mechanism to anticipate the next item. The framework empowers item representations with external semantic knowledge to offer in-depth contextual insights over and above the conventional interaction-based modeling. Substantial experiments on four benchmark datasets, i.e., MovieLens 1M, MovieLens 20M, Goodbooks-10k and Anime, show steady gains on a variety of ranking measures, including Precision at K, Recall at K, Mean Reciprocal Rank (MRR), Hit Rate (HR at K) and NDCG at K, over strong baseline models. In addition, the proposed framework is evaluated through an initial federated learning feasibility study using the standard Federated Averaging (FedAvg) algorithm to investigate its applicability in federated recommendation feasibility scenarios. The federated experiments are intended as a proof-of-concept rather than a comprehensive benchmark of practical federated recommender systems. The results demonstrate that the proposed Hybrid Transformer–LLM framework maintains competitive recommendation performance under decentralized training while preserving user data locality. Furthermore, the integration of LLM-derived semantic representations consistently improves recommendation quality across sparse and content-rich datasets with only a modest computational overhead. These findings demonstrate that semantic-enhanced Transformer recommendation can be effectively extended to decentralized learning environments while providing a scalable foundation for future privacy-preserving recommender systems.