Personalized Recommendation Agents with Self-Consistency

This paper presents the 2nd place solution of the WWW'25 AgentSociety Challenge (Recommendation Track), which aims to advance the development of Large Language Model (LLM)-based agents for personalized recommendation. During this challenge, participants are tasked with building LLM-based agents capable of ranking items according to user preferences. To evaluate the practical capabilities of these agents, the organizer provides a robust evaluation platform that faithfully simulates real-world recommendation scenarios across multiple domains (e.g., Yelp, Goodreads, and Amazon). To tackle this challenge, our team, RecHackers, develops an agent-voting system inspired by the idea that self-consistency across multiple LLM outputs can enhance the accuracy. Specifically, our approach follows a prompting-sampling-voting paradigm. We first carefully design prompts for individual agents based on user interaction and item characteristics. Then, we query LLM-based agents multiple times with a controlled temperature to obtain a diverse range of candidate rankings. Lastly, we aggregate these rankings with predefined voting mechanisms, such as majority voting or Borda count, to derive the final ranking list. This simple yet effective solution achieved the 2nd place in the WWW'25 AgentSociety Challenge (Recommendation Track).