Dataset Pruning Using Early Exit Networks

We present EEPrune, a novel dataset pruning algorithm that leverages early exit networks during training. EEPrune utilizes the innate ability of early exit networks to assess the difficulty of individual samples and avoid overthinking. It applies multiple criteria to decide whether to prune them. Specifically, for a training sample to be discarded, the confidence level of the model at the early exit should be above a certain threshold, along with a correct classification at both the early exit and final layers. Extensive experiments and ablations on CIFAR-10, CIFAR-100, Tiny ImageNet, KMNIST, and ImageNet datasets demonstrate that EEPrune consistently outperforms other dataset pruning methods.