Enhancing Video Surveillance Applications Through a Comparative Evaluation of Deep Learning-Based Object Detection and Tracking

Advancements in intelligent video surveillance are increasingly driven by deep learning algorithms for visual object detection and tracking. This paper presents a comprehensive benchmarking study of integrated detection and tracking pipelines for vehicle and pedestrian surveillance applications. We systematically evaluate two-stage (Faster R-CNN) and single-stage (YOLOv7, YOLOv8, YOLOv11, YOLOv12) detectors, integrated with DeepSORT, ByteTrack, BoT-SORT, and OC-SORT trackers within a tracking-by-detection framework. To ensure cross-domain robustness of detection models, the study leverages both the BDD100K benchmark (for detection and tracking evaluations) and a newly curated Ethio-AA dataset (for detection evaluation and cross-domain generalization assessment of detection models). Quantitative results demonstrate that Faster R-CNN achieves competitive accuracy, while YOLOv12 provides a better trade-off, achieving 94.5% [email protected] for vehicles at 55 FPS. In terms of tracking, ByteTrack exhibits the most consistent performance on both vehicle and pedestrian cases, achieving MOTA of 92.5% for vehicles and 84.2% for pedestrians, with good robustness to low-confidence detections. The camera motion compensation mechanism in BoT-SORT helps it preserve the highest identity (IDF1: 67.1%). OC-SORT also performs competitively in dense scenes. Through systematic sensitivity analysis of the association parameter λ, we empirically determine recommended values (λ = 0.7 for vehicles, λ = 0.5 for pedestrians) based on BDD100K validation set evaluation. Our results demonstrate that YOLOv12+ByteTrack achieves the best speed–accuracy trade-off for real-world video surveillance, while YOLOv12+BoT-SORT is optimal for applications prioritizing identity preservation. This benchmarking study provides empirical guidance for surveillance practitioners, establishing performance baselines and configuration recommendations across five detectors, four trackers, two datasets, and category-specific scenarios.