Software-Hardware Exploration of Early-Exit Neural Networks on Edge Accelerators

In this paper, we propose a Software-Hardware exploration framework for Early-Exit (EE) Neural Networks (SHEN), to optimally explore the configurations of an EE network on customizable edge accelerators. consists of two parts: a newly proposed method to optimally determine the number and positions of intermediate classifiers for an EE network, and a genetic-based method to explore the optimal configuration of an accelerator for the network. We can utilize to generate a series of design points for an EE network and accelerators, allowing for the selection of a suitable design configuration for a specific requirement. Experimental results show that finds better EENN configurations with increased accuracy by up to 5% compared to its backbone counterpart and state of the art. Our exploration also shows its effectiveness in reducing energy-delay product (EDP), where reduces EDP by up to 85% and 41% compared to its backbone counterpart and state of the art. Our code is available at https://github.com/qianying-gong/SHEN.