IB-AMG: Aircraft Mission Generation With Inference-based Vision-Language-Action Model

Addressing challenges in Unmanned Aerial Vehicle (UAV) mission generation, particularly how current Large Language Model (LLM) methods struggle to create stable and efficient paths for complex spatial tasks, this study proposes an Inference-Based Aircraft Mission Generation (IB-AMG) framework. IB-AMG integrates language, vision, and action models, crucially leveraging the strong logical reasoning and planning capabilities of Inference-Based LLMs. Importantly, it uses an approach where LLMs plan paths using simpler, relative coordinates (P<inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</inf>) from a Vision-Language Model (VLM), avoiding the difficulties caused by complex, high-precision geographic data (P<inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">g</inf>). Combined with carefully designed prompts, which also limit overly complex thinking, IB-AMG creates mission plans from text instructions without prior examples (zero-shot setting). Experimental results demonstrate that compared to baseline methods relying on generative LLMs, IB-AMG achieves mission generation efficiency comparable to human experts while exhibiting significantly superior stability and effectively avoiding suboptimal or unreasonable paths. This study shows that using strong reasoning models combined with customized inputs (P<inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</inf>) and guided thinking (constrained prompts) leads to reliable UAV mission planning. It offers a promising approach for developing efficient and reliable autonomous UAV systems via multi-modal reasoning.