ToolSpectrum: Towards Personalized Tool Utilization for Large Language Models

While integrating external tools into large language models (LLMs) enhances their ability to access real-time information and domainspecific services, existing approaches focus narrowly on functional tool selection following user instructions, overlooking the contextaware personalization in tool selection.This oversight leads to suboptimal user satisfaction and inefficient tool utilization, particularly when overlapping toolsets require nuanced selection based on contextual factors.To bridge this gap, we introduce ToolSpectrum, a benchmark designed to evaluate LLMs' capabilities in personalized tool utilization.Specifically, we formalize two key dimensions of personalization, user profile and environmental factors, and analyze their individual and synergistic impacts on tool utilization.Through extensive experiments on ToolSpectrum, we demonstrate that personalized tool utilization significantly improves user experience across diverse scenarios.However, even state-of-theart LLMs exhibit the limited ability to reason jointly about user profiles and environmental factors, often prioritizing one dimension at the expense of the other.Our findings underscore the necessity of context-aware personalization in tool-augmented LLMs and reveal critical limitations in current models.Our data and code are available at https://github.com/ BUAA-IRIP-LLM/ToolSpectrum. C Prompt Templates Prompt Template for Generating Consumption PreferenceYour current task is to generate a natural language description based on the provided purchase behaviour preference keyword.The description should explain the user's buying habits, preferences, or decisionmaking style in a realistic and specific way.Example: Keyword: Cost-effectiveness Output: Prefers to buy cost-effective products and is highly sensitive to prices.Keyword: Promotions Output: Frequently shops during promotional events and enjoys comparing prices to find the best deals.