Gemini+: Enhancing Real-Time Video Analytics With Dual-Image FPGAs

Real-time video analytics demand intensive computing resources, often exceeding device capabilities. Heterogeneous computing resources like CPU and GPU, usually work collaboratively to ensure real-time performance. GPU manage data-intensive computing, while CPU handle instruction-intensive tasks. However, analytics accuracy can degrade because of the imbalance between CPU-GPU workloads and resources, and static resources in most systems limit adaptability to dynamic workloads. In addition to CPU-GPU resource control, accuracy is highly dependent on video analytics configuration including resolution, frame rate, and model selection. In this paper, we propose Gemini+, a hardware-accelerated video analytics pipeline empowered by dual-image FPGAs. It enables flexible CPU-GPU resource adaptation by providing near-instantaneous switching between two pre-configured images. We investigate CPU-GPU resource control and video analytics configuration adaptation in a dual-image FPGA-based video analytics pipeline. We study and formulate two problems of practical importance, a single-camera and multi-camera dual computing resource control problem, i.e., SC-DCRC and MC-DCRC. We analyze the problem complexity and develop optimal and sub-optimal algorithms. We evaluate our algorithms through simulation and build a prototype for verification. The results show that Gemini+ can improve analytic accuracy by 25% compared to fixed CPU-GPU resource systems and 88% compared to GPU-dominant systems.