A Taxonomy of Failures in Tool-Augmented LLMs

Large language models (LLMs) can perform a variety of tasks given a user prompt that contains a description of the task. To enhance the performance of LLMs, recent research has focused on augmenting LLMs with external tools, such as Python APIs, REST APIs, and other deep learning models. Much of the research on tool-augmented LLMs (TaLLMs) has focused on improving their capabilities and accuracy. However, research on understanding and characterizing the kinds of failures that can occur in these systems is lacking. To address this gap, this paper proposes a taxonomy of failures in TaLLMs and their root causes, details an analysis of the failures that occur in two published TaLLMs (Gorilla and Chameleon), and provides recommendations for testing and repair of TaLLMs.