Latency-aware DNN Inference Acceleration with Dynamic Model Partitioning in Vehicular Edge Computing

Distributed embedded inference architectures play a critical role in realizing intelligent decision-making for vehicles. Deep Neural Networks (DNNs) are a pivotal technology for driver assistance. During the operation of autonomous vehicles, sensors generate a massive amount of data, necessitating the rapid computation of DNN tasks. However, the computational resources and energy available locally on the vehicle are limited, making it challenging to respond swiftly to these tasks. Edge collaborative inference integrates edge computing with DNN task inference, harnessing the powerful computational capabilities of edge servers to complete the inference within tolerable latency. Scholars have employed model partitioning to accelerate edge collaborative inference. Nevertheless, research on model partitioning under local energy consumption constraints in vehicle networks is scarce, and few studies consider local vehicle constraints. Addressing the high computational demands of local DNN tasks, and the latency sensitivity that complicates DNN inference, this article proposes a method for collaborative DNN task inference based on model partitioning. In contrast to existing work, this study considers vehicle’s computational energy consumption and memory constraints, formulating a DNN model partitioning problem to minimize latency. The problem of DNN model partitioning under constraints is transformed into a maximum flow problem in a constrained Directed Acyclic Graph (DAG) network flow graph, and a DNN Model Partitioning Algorithm Based on Dinic’s Algorithm Under Constraints(MPAUC) is proposed for the solution. Compared to the DADs method, our approach achieves approximately a 1.5× reduction in latency and a 1.2× increase in throughput on the InceptionV3 network. These results demonstrate the superiority of the MPAUC algorithm in optimizing the deployment of deep learning models.