Vision‐Based UAV Detection and Tracking Using Deep Learning and Kalman Filter

ABSTRACT The rapid increase in unmanned aerial vehicles (UAVs) usage across various sectors has heightened the need for robust detection and tracking systems due to safety and security concerns. Traditional methods like radar and acoustic sensors face limitations in noisy environments, underscoring the necessity for advanced solutions such as deep learning‐based detection and tracking. Hence, this article proposes a two‐stage platform designed to address these challenges by detecting, classifying, and tracking various consumer‐grade UAVs. The tracking efficacy of the proposed system is assessed using a combination of deep learning and Kalman filter techniques. Specifically, we evaluate models such as YOLOv3, YOLOv4, YOLOv5, and YOLOx to identify the most efficient detector for the initial detection stage. Moreover, we employ both the Kalman filter and the Extended Kalman filter for the tracking stage, enhancing the system's robustness and enabling real‐time tracking capabilities. To train our detector, we construct a dataset comprising approximately 10,000 records that capture the diverse environmental and behavioural conditions experienced by UAVs during their flight. We then present both visual and analytical results to assess and compare the performance of our detector and tracker. Our proposed system effectively mitigates cumulative detection errors across consecutive video frames and enhances the accuracy of the target's bounding boxes.