Energy-Efficient, Delay-Constrained Edge Computing of a Network of DNNs

This paper presents a novel approach for executing the inference of a network of pre-trained deep neural networks (DNNs) on commercial-off-the-shelf devices that are deployed at the edge. The problem is to partition the computation of the DNNs between an energy-constrained and performance-limited edge device <inline-formula><tex-math notation="LaTeX">$\boldsymbol{\mathcal{E}}$</tex-math></inline-formula>, and an energy-unconstrained, higher performance device <inline-formula><tex-math notation="LaTeX">$\boldsymbol{\mathcal{C}}$</tex-math></inline-formula>, referred to as the <i>cloudlet</i>, with the objective of minimizing the energy consumption of <inline-formula><tex-math notation="LaTeX">$\boldsymbol{\mathcal{E}}$</tex-math></inline-formula> subject to a deadline constraint. The proposed partitioning algorithm takes into account the performance profiles of executing DNNs on the devices, the power consumption profiles, and the variability in the delay of the wireless channel. The algorithm is demonstrated on a platform that consists of an NVIDIA Jetson Nano as the edge device <inline-formula><tex-math notation="LaTeX">$\boldsymbol{\mathcal{E}}$</tex-math></inline-formula> and a Dell workstation with a Titan Xp GPU as the cloudlet. Experimental results show significant improvements both in terms of energy consumption of <inline-formula><tex-math notation="LaTeX">$\boldsymbol{\mathcal{E}}$</tex-math></inline-formula> and processing delay of the application. Additionally, it is shown how the energy-optimal solution is changed when the deadline constraint is altered. Moreover, the overhead of decision-making for our proposed method is significantly lower than the state-of-the-art Integer Linear Programming (ILP) solutions.