Inference Routing Over Multi-Hop Edge Networks

The emergence of artificial intelligence (AI) has paved the way for ubiquitous intelligent application on edge devices. However, the deep neural networks (DNN) based AI technology suffers from heavy computing burden, which is prohibitive for edge devices as they have limited energy and computing capability. Toward this end, this paper focuses on the multi-splitting of DNN over wireless multi-hop networks so as to distribute the computing over multiple network nodes for achieving efficient edge inference. The challenge is how to choose an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">inference routing</i> in which both the transmission and inference can be efficiently relayed hop-by-hop. We propose a DNN multi-splitting method along with dynamical early-exit of inference based on communication and computation conditions. We formulate an optimization problem of joint routing, split points selection, and model deployment to minimize the end-to-end inference latency. The backward induction approach is proposed to transfer latency information among network nodes to find the optimal solutions. The experimental results demonstrate the superiority of our work in reducing inference latency.