Real-Time Collaborative Edge Ai Inference: Joint Dnn Splitting and Batch Optimization

Cloud-edge collaboration has become a pivotal approach in edge computing, especially for applications needing real-time inference. This holds particular significance in scenarios where continuous and concurrent inference tasks have stringent latency demands. This paper considers a scenario where dynamically arriving tasks can be partially offloaded to a server with parallel computing for inference. We build a dynamic model for each task's inference process and formulate the problem of minimizing the long-term delay violation rate. To address the strategy problem of joint splitting and batch scheduling for multiple users, we adopt a sequential decision-making approach which effectively lowers the computational complexity. Meanwhile, we innovatively devise a wireless resource pool and a batch scheduling linked list to characterize the system's overall computing and communication resources. On this basis, we can evaluate different task splitting and batch scheduling strategies to make more effective decisions. Simulation results demonstrate a significant improvement in latency guarantees with millisecondlevel execution efficiency and robust scalability across diverse user scales, outperforming baseline methods under heterogeneous workloads.