Combining Early Exit and Selective Prediction for Convolutional Neural Networks

The deployment of CNN in real-time and resource-constrained applications poses critical challenges due to their computational demands and the need for reliable decision making. In this paper, we combine adaptive inference via Early Exits (EE) with Selective Prediction (SP) to address these challenges. Early exits allow confident predictions at intermediate layers, while selective prediction introduces uncertainty estimation modules, enabling the system to abstain from low-confidence decisions or continue inference through deeper layers. This combined design lowers the risk of overconfident but erroneous predictions and improves the trade-off between performance and accuracy. As a case study, we implement and evaluate our approach on a real-time traffic sign detection task, processing the input of an RGB camera in the forward direction. In this paper, we demonstrate improved performance compared to baseline models. Compared to SP-only (Selective Prediction) and EE-only (Early Exit) baselines, our hybrid model achieves low inference depth (1.20), leading to reduced computational demand and latency. Despite this efficiency, the model maintains a high prediction accuracy (90.3%) and a low abstention rate (1.6%), ensuring fast and reliable decision making suitable for time-critical embedded applications. This demonstrates an effective trade-off between effective computation and predictive reliability.