Efficient and Responsible Adaptation of Large Language Models for Robust Top-k Recommendations

Conventional recommendation systems (RSs) often optimize for aggregate accuracy, inadvertently underserving users with sparse interaction histories. Although large language models (LLMs) exhibit strong zero- and few-shot ranking capabilities, their use in RS pipelines faces a two-fold challenge: (i) scalability and generalizability : since many evaluations rely on small, randomly sampled user subsets that limit generalization to real-world populations; and (ii) responsible adaptation under resource constraints : since LLMs impose non-trivial cost and latency that preclude indiscriminate use. To address these challenges, we propose a hybrid task allocation framework that proactively allocates ranking tasks between traditional RSs and LLMs to enhance robustness and efficiency. Our strategy works by first identifying weak and inactive users who receive suboptimal ranking performance from RSs. Next, we use an in-context learning approach for such users, wherein each user’s interaction history is contextualized as a distinct ranking task. We evaluate our hybrid framework that integrates eight diverse recommendation algorithms, three popular datasets, and three LLMs (both open- and closed-source), demonstrating that it significantly reduces weak users by approximately 12% while maintaining cost-effectiveness through targeted LLM utilization.