A Deep Reinforcement Learning Framework for Last-Mile Delivery with Public Transport and Traffic-Aware Integration: A Case Study in Casablanca

Optimizing last-mile delivery operations is an essential component in making a modern city livable, particularly in the face of rapid urbanization, increasing e-commerce activity, and the growing demand for fast deliveries. These factors contribute significantly to traffic congestion and pollution, especially in densely populated urban centers like Casablanca. This paper presents an innovative approach to optimizing last-mile delivery by integrating public transportation into the logistics network to address these challenges. A custom-built environment is developed, utilizing public transportation nodes as transshipment nodes for standardized packets of goods, combined with a realistic simulation of traffic conditions through the integration of the travel time index (TTI) for Casablanca. The pickup and delivery operations are optimized with the proximal policy optimization algorithm within this environment, and experiments are conducted to assess the effectiveness of public transportation integration and three different exploration strategies. The experiments show that scenarios integrating public transportation yield significantly higher mean rewards—up to 1.49 million—and more stable policy convergence, compared to negative outcomes when public transportation is absent. The highest-performing configuration, combining PPO with segmented training and public transport integration, achieves the best value loss (0.0129) and learning stability, albeit with a trade-off in task completion. This research introduces a novel, scalable reinforcement learning framework to optimize pickup and delivery with time windows by exploiting both public transportation and traditional delivery vehicles.