PLAY2PROMPT: Zero-shot Tool Instruction Optimization for LLM Agents via Tool Play

Large language models (LLMs) are increasingly integrated with specialized external tools, yet many tasks demand zero-shot tool usage with minimal or noisy documentation.Existing solutions rely on manual rewriting or labeled data for validation, making them inapplicable in true zero-shot settings.To address these challenges, we propose PLAY2PROMPT, an automated framework that systematically "plays" with each tool to explore its input-output behaviors.Through this iterative trial-and-error process, PLAY2PROMPT refines tool documentation and generates usage examples without any labeled data.These examples not only guide LLM inference but also serve as validation to further enhance tool utilization.Extensive experiments on real-world tasks demonstrate that PLAY2PROMPT significantly improves zeroshot tool performance across both open and closed models, offering a scalable and effective solution for domain-specific tool integration 1 .

Authors

Publication Details

Published
2025-01-01
DOI
https://doi.org/10.18653/v1/2025.findings-acl.1347
Primary Topic
Multi-Agent Systems and Negotiation
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

PLAY2PROMPT: Zero-shot Tool Instruction Optimization for LLM Agents via Tool Play

Yang Zhang, Kaizhi Qian, Fang Wei, Yingchun Zhu et al.
Multi-Agent Systems and Negotiation
article

PLAY2PROMPT: Zero-shot Tool Instruction Optimization for LLM Agents via Tool Play

Yang Zhang, Kaizhi Qian, Fang Wei, Yingchun Zhu, James Glass
article en

Abstract

Large language models (LLMs) are increasingly integrated with specialized external tools, yet many tasks demand zero-shot tool usage with minimal or noisy documentation.Existing solutions rely on manual rewriting or labeled data for validation, making them inapplicable in true zero-shot settings.To address these challenges, we propose PLAY2PROMPT, an automated framework that systematically "plays" with each tool to explore its input-output behaviors.Through this iterative trial-and-error process, PLAY2PROMPT refines tool documentation and generates usage examples without any labeled data.These examples not only guide LLM inference but also serve as validation to further enhance tool utilization.Extensive experiments on real-world tasks demonstrate that PLAY2PROMPT significantly improves zeroshot tool performance across both open and closed models, offering a scalable and effective solution for domain-specific tool integration 1 .

Innovation and Technology Commission
Openalex Percentile: Top 90%
Multi-Agent Systems and Negotiation
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.