Common DNN Layer Sharing aware Task Scheduling for Inference Acceleration in Serverless Edge Computing

Serverless edge computing, characterized by fine-grained resource allocation and rapid task scheduling, is effectively implemented in edge clouds to support a diverse array of Deep Neural Network (DNN) based inference tasks. However, before executing inference tasks, the system needs to load DNN models into a container, a process known as cold start. The cold start introduces significant latency, thereby prolonging the task completion time. In particular, model loading time constitutes approximately 50%–70% of the overall task lifecycle, making it comparable to the duration of inference execution. Fortunately, we notice that some layers are required by multiple models and only need to be loaded once if these tasks are scheduled onto one server, i.e., DNN layer sharing. However, the computational resource differences among edge servers and their limited memory capacity make task scheduling and layer loading decisions particularly challenging. Therefore, to fully leverage the potential of DNN layer sharing, we investigate the Layer Sharing aware Task Scheduling (LSTS) problem with the goal of minimizing the task completion time. We formulate it into a Quadratic Integer Programming (QIP) problem and linearize it into an Integer Linear Programming (ILP) form, which is then proved as NP-hard. To tackle the computation complexity, we propose a Randomized Rounding-based Layer-Sharing-Aware Task Scheduling algorithm (LSTS-RR). Through comprehensive experimental evaluations, we confirm the effectiveness of our algorithm, as it reduces task completion time by more than 36% compared to other state-of-the-art approaches across a range of widely recognized DNN models.