TypeFly: Low-Latency Drone Planning With Large Language Models

Recent advancements in robot planning using large language models (LLMs) have demonstrated significant potential, primarily due to LLMs' capabilities to understand natural language commands and generate executable plans in various languages. However, in time-sensitive and interactive applications involving mobile robots, particularly drones, the sequential token generation process inherent to LLMs introduces substantial latency, i.e., response time, during the control plan generation. In this paper, we present a system called ChatFly that tackles this latency problem using a combination of a novel programming language called MiniSpec and its runtime to reduce both the response time and generation time for the robot plan. That is, instead of asking an LLM to write a program (robotic plan) in the popular but verbose Python, ChatFly gets it to do it in MiniSpec specially designed for token efficiency and stream interpreting. Using a set of challenging drone tasks, we show that design choices made by ChatFly can reduce the average response time to 74% compared to existing works and provide a more consistent user experience, enabling responsive and intelligent LLM-based drone control.