Q-Learning-Based Multi-Objective Path Planning for AAV Parcel Delivery Through Public Transportation Vehicles

Combining Autonomous Aerial Vehicles (AAVs) with public transportation systems offers a novel approach to enhancing urban last-mile delivery efficiency. However, current research lacks comprehensive solutions for optimizing AAV path planning that can adapt to real-time changes in public transportation schedules while avoiding dynamically changing high-interference zones, ensuring consistent drone connectivity, and accounting for limited drone battery life. This paper addresses these gaps by optimizing and balancing multiple objectives, such as ensuring early arrival, minimizing energy consumption, and reducing signal interference. To achieve these goals, we propose a Q-learning-based algorithm for real-time path optimization, ensuring path adaptability to random changes in interference zones and public transportation schedules, and compare its performance against multi-objective A* algorithm variants. Results show that Q-learning achieves superior early arrival times, energy efficiency, and reduced interference, with a nearly 100% success rate in package deliveries, compared to 60-80% for A*. These findings demonstrate the robustness and reliability of Q-learning in AAV-Aided Public Transportation systems, making it a valuable solution for real-world urban delivery challenges.