Deep Reinforcement Learning for Collaborative Inference in Local-Serverless Edge Computing

High-dimensional parametric models and large-scale mathematical computations limit the efficiency of IoT devices. The emergence of serverless edge computing provides a solution where users can deploy models as serverless functions and delegate provisioning and scaling to the platform. However, the resource-constrained nature of edge resources leads to their inefficiency or inability to serve large neural networks. Therefore, this paper emphasizes collaborative inference between devices and serverless edge computing for DNN models. Specifically, we consider the partitioning and offloading of DNN models and propose a DRL-based strategy to learn the joint decision of model partitioning and function memory type selection to achieve a cost-optimal service that meets the SLO requirements. In addition, we introduce a simulated annealing algorithm in deep reinforcement learning that focuses on exploration in the early stage and empirical value in the later stage. Finally, experimental results show that our algorithm can satisfy various SLOs at a low service cost and outperform three benchmarking strategies.