A Hybrid Method for Real-Time Vehicle Tracking from a Moving Camera with YOLO and KLT

Deep neural detectors such as YOLO reliably detect vehicles, but running them on every frame is computationally expensive for real-time CPU performance.We propose a hybrid tracking method that requires no additional training.The detector operates only on every 𝐾𝐾 keyframe, while intermediate bounding boxes are propagated using Kanade-Lucas-Tomasi (KLT) feature point tracking with forward-backward error validation.Since tracked points belong directly to the vehicles, the method inherently handles camera movement without requiring prior video stabilization.Implemented in MATLAB using YOLOv3, the approach was evaluated on a challenging night video featuring wet asphalt, headlight reflections, and camera displacement up to 190 px.At a detection interval of 𝐾𝐾 = 5, the hybrid method achieves MOTA 0.73 and IDF1 0.76 (compared to 0.69 and 0.74 for full-frame YOLOv3), reduces identity switches from 55 to 13, and yields a nearly 4-fold speedup (9.3 fps vs. 2.4 fps).At 𝐾𝐾 = 15, KLT maintains an IDF1 of 0.62, outperforming simple frame holding (0.38).