UAV-Based Vehicle Detection and Tracking in Urban Environments Using Multi-Task CNN and Deep Reinforcement Learning

This paper presents a real-time vehicle detection and tracking system using an unmanned aerial vehicle (UAV) to address challenges in dynamic urban environments. The system combines a convolutional neural network (CNN) for vehicle detection with a deep Q-network (DQN)-based navigation policy for continuous tracking. Input images are enhanced using contrast limited adaptive histogram equalization (CLAHE) and unsharp masking. The CNN jointly predicts vehicle center coordinates and probabilistic heatmaps, while a self-attention module captures long-range spatial dependencies to improve detection under clutter and occlusion. The DQN is trained on multi-step spatiotemporal states to learn optimal UAV movement strategies under diverse weather and structural conditions. Experiments conducted in a three-dimensional (3D) urban simulation environment using Unity’s machine learning agents (ML-Agents) show that the self-attention design reduced pixel-level localization error by about 7%, and the DQN-based tracking policy achieved stable convergence after approximately 2000–3000 episodes. These results demonstrate high tracking accuracy and system stability, highlighting the potential of the proposed approach for real-world UAV-based traffic monitoring applications.2018 The Korean Institute of Communications and Information Sciences. Publishing Services by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).