GeoPet: Interactive Prompt Engineering for Enhancing Tool Calling of Large Language Models in Geospatial Tasks

Geospatial tasks often require the coordination of various spatial algorithms and operations, which are usually performed through tool calling guided by natural language prompts. Crafting effective prompts is challenging due to the inherent complexity and ambiguity of natural language. In this paper, we present GeoPet, a visual analytics system designed to simplify the process of prompt engineering to improve the performance of geospatial tool callings on large language models (LLMs). At its core, GeoPet is a sophisticated tool recommendation method that accepts geospatial tasks as input, decomposes them into atomic tasks, identifies relevant tools, and extracts the most relevant tool descriptions for these atomic tasks. The system is designed to support interactive prompt engineering of geospatial tool invocations, enabling users to explore the connections between geospatial tasks and established tools and to evaluate their performance. This enables crafting and refining prompts that coordinate LLMs with human expertise. GeoPet's satisfaction, practicality, usability, and visual design are validated through two case studies and a user study. These demonstrate that the system significantly eases the burden of rapid engineering and skillfully guides LLMs in geospatial tool calling capabilities. By providing a visual and interactive system for prompt engineering, GeoPet helps users navigate complex geospatial tasks and improves the overall efficiency and accuracy of tool calling for LLMs.