UAV Visual Path Planning Using Large Language Models

Unmanned Aerial Vehicles (UAVs) heavily rely on Global Positioning Systems (GPS) for navigation, limiting their functionality in indoor GPS-denied environments. This paper investigates the application of Large Language Models (LLMs) for visual path planning in such scenarios. This work proposed a new LLM-based approach for understanding visual data captured by the UAV’s camera. By analyzing this data in terms of the positions of the detected persons and depth information, the fine-tuned LLM would generate safe and efficient flight paths. To validate the proposed approach, we have created an indoor virtual navigation environment for the entrance of our center (JRC-AI) with 3 standing persons and 2 randomly moving. Guided by LLMs, the mission of UAVs is to reach the target goals that result in the minimum collisions. The reported results clearly showed that the proposed LLMs achieved better results than the standard deep reinforcement learning DQN model in both the average number of collisions as well as the traveled distance toward the goal point.