Weight-Shifting Based Pruning for Convolutional Neural Networks

The paper presents an adaptation of the novel self-recovering technique known as weight-shifting for application in the pruning of nodes from the fully-connected layers of convolutional neural networks. The proposed methodology involves a binary search strategy to identify the optimal subset of nodes for removal, facilitating efficient redistribution of node weights using the Moore-Penrose pseudo-inverse. The proposed methodology was evaluated by across multiple data sets, including CIFAR-100, Food101, Stanford Cars, and Flowers102. The experimental results demonstrated that the proposed methodology is able to maintain high accuracy and top-k categorical accuracy performance with minimal reductions even after pruning a significant percentage of nodes. The paper contributes to neural network compression and optimization by providing a versatile solution that enhances model efficiency without significantly compromising performance, offering potential benefits for applications in resource-constrained environments.