Hardware Partitioning for Vision Analytics-based Systems: Performance and Trade-offs

Mobile Vision Analytics (MVA) is essential for advancing applications, which analyze visual data using deep neural network (DNN) models to extract contextual insights. While cloud based approach is common, the high latency incurred in transmission leading to a shift toward edge computing. However, the limited resource on edge leads to the challenge of concurrent DNN execution on shared GPU. This paper proposes a novel mathematical framework that models MVA workflows over an edge infrastructure, including the partitioning of GPU resources for concurrent DNNs. Our model integrates local and remote processing steps, accounting for wireless transmission and GPU sharing, and provides a structured approach to estimating per-step time consumption. A detailed numerical evaluation demonstrates the latency and resource utilization of each part of the system under varied scenarios, offering insights for the design of latency-sensitive MVA solutions with concurrent models on shared edge GPU infrastructure.