Flow Control Solution to Avoid Bottlenecks in Edge Computing for Video Analytics

In this article we present a new approach to scaling edge server based real-time video analytics by utilizing a novel flow control mechanism that we call ‘Pace Steering’ (PS). In contrast to server-side scheduling, flow control enables the server to control the frame rate of the connected streams, thus extending its control to also the network traffic. By exploiting the fact that video analytics applications have a constant frame rate for each client, and a predictable inference time for the frame processing, Pace Steering is able to avoid server-side queueing and balance the network load in a shared wireless access link, leading to a better utilization of both the compute and the network resources. We provide a mathematical analysis to show how to synchronize video streams with delays based on server state information. We then show how queueing time of a request provides an ideal synchronization delay, and then extend this idea to consider batching for higher throughput. We evaluate our approach with benchmarks in a physical testbed using a commodity Wi-Fi. The results show that PS enables up to 80 concurrent 10 frames per second (FPS) streams to be served without latency requirement violations for $\mathbf{9 5 \%}$ of the sent frames, which is twice as many streams as without PS.