A Channel Pruning Method Using Subtraction-Average-Based Optimizer

Convolutional Neural Networks(CNNs) require huge resources, which hindered the implement on light devices. However, existing pruning techniques primarily concentrate on choosing channels based on their importance or on regularization strategies that are crafted through empirical knowledge, the compressed networks obtained through these methods have unstable performance and require unnecessary human intervention. This paper proposes a new channel pruning method based on the Subtraction-Average-Based Optimizer, dubbed as SABOPruner. To facilitate deployment on FGPA, we constrain the pruning step size of each layer to be a multiple of 8. Finally, the search was transformed into an optimization problem and we solve it by integrating the SABO algorithm, reducing the need for manual intervention. We introduced Adaptive Batch Normalization to improve the efficiency of fine-tuning, this strategy also accelerates the search efficiency of the SABO algorithm. SABOPruner has been demonstrated to be effective, we conducted efficacy tests on four leading CNN models using two image classification datasets. On the ILSVRC-2012 dataset, the pruned ResNet-50 model reduced 55.15% in FLOPs and make only a 0.9% decrease in accuracy compared to the baseline. With MobileNetV2, we achieve a 51%-FLOPs reduction with a 3.05% performance higher on ImageNet-1K.