CATALYSE: Communication-aware Task Mapping for DNNs on Multicore Systems at Edge

Deep Neural Networks (DNN) are being used in several applications on edge devices in real time. Moreover, multiple DNNs rather than a single DNN are executed at the same time to perform different modalities of a task. State-of-the-art edge devices typically consist of multiple processing units on-chip. Therefore, DNNs must be intelligently scheduled at runtime to harness the capability of multiple processing elements at the fullest. Communication between different processing elements and the memory constitute significant portion of the total latency of DNN execution. Although there are several state-of-the-art DNN scheduling and mapping policies which aim to reduce latency of DNN execution, they do not take communication into account to minimize their execution time. Therefore, in this work we present communication-aware task scheduling for DNNs on multicore systems. Specifically, we profile different DNN layers to obtain communication latency between them and employ a greedy algorithm at runtime to map the DNN layers onto available processing elements on a system. Extensive experimental evaluations with various DNNs used at real time show that our proposed technique is able to minimize the execution latency by up to 46.60% with respect to state-of-the-art DNN mapping technique.