CADI: Carbon-Aware Distributed Inference

This work proposes an integer linear program (ILP) formulation for Carbon-Aware Distributed Inference (CADI). We provide a model for carbon footprint (CF) for distributed inference which depends on factors such as energy consumption due to computation of DNN layers, communication between edge devices, and idle energy when an edge device is on hold to receive the results of another device. CADI assigns the layers of a Deep Neural Network (DNN) to collaborating edge devices which belong to different power grids with different carbon footprints in the same time period to minimize the overall CF. In our experiments, we show the conventional objective of minimizing energy results in on-average 3.28X higher CF, compared to our proposed objective of directly minimizing CF itself. Compared to single-device inference, we show executing same number of inferences on a single device (connected to grid with higher carbon intensity) in the same duration of time may result up to 1.4X higher CF compared to CADI deciding when to transition to a second device (connected to a grid of lower carbon intensity).