RACENet: Real-time Adaptive Class-aware Early-exit Networks for Edge Devices

As the integration of edge and IoT devices continues to surge, the need for streamlined machine learning solutions, notably Deep Neural Networks (DNNs), becomes paramount. However, the inherent computational demands and significant memory requirements of DNNs present hurdles for seamless edge integration. Early-Exit Networks (EENs) have emerged as a solution by adding early exits to DNNs, allowing for early inference and facilitating dynamic deployment on devices with varying capabilities. Despite their advantages, current EEN models uniformly handle samples from all classes, which is suboptimal in many edge scenarios where different classes require varying processing speeds and response times. Our research introduces RACENet, a novel architecture with adaptive early-exit capabilities that adjusts class prioritization in real-time. This flexibility enables RACENet to respond to evolving class priorities during runtime. Extensive evaluations on vision tasks, and network intrusion detection demonstrate that RACENet maintains accuracy while dynamically managing class priorities and accelerating inference for high-priority classes, all without adding additional computational burdens on constrained edge devices.