EdgePrios: Joint Scheduling of Initialization and Execution for Serverless Inference Acceleration in Edge Cloud

The rapid deployment of intelligent applications on edge cloud calls for efficient and responsive DNN inference, especially under the burst scenarios of inference request. Serverless inference offers a promising solution by enabling rapid and flexible activation of inference tasks to cope with peak request, but its achievable performance is highly influenced by the initialization overhead. Existing studies on inference acceleration mainly focuses on execution optimization, they usually overlook the fact that inference performance also heavily depends on the initialization. In this paper, we propose EdgePrios, a novel priority-based scheduling mechanism that jointly optimizes initialization and execution phases for serverless inference acceleration. EdgePrios dynamically prioritizes tasks by considering workloads, dependency relationships, and the current status of available resources. It enables precise assignment of tasks to computing resources while minimizing overall inference time in edge cloud. Extensive trace-driven evaluations demonstrate the efficiency of EdgePrios as it outperforms state-of-the-art methods, achieving 15.6%-38.8% reduction in inference time under varying resource configurations, network bandwidths, and application topologies.