Structured pruning for group regularized convolutional neural networks via dynamic regularization factor

High demand for computation and storage largely hinders the deployment of deep convolutional neural networks (CNNs) in resource constrained devices. Regularization-based pruning methods are effective for the model compression of CNNs by introducing a regularization term into the loss function. However, such methods typically compress weights to zero utilising a fixed regularization factor, which cannot impose different punishments on weights or structures with different importance. In this paper, a group regularization method is proposed to impose a dynamic regularization factor to each filter of convolutional layers, named GRDRF. The scaling factors of batch normalization (BN) layer are used to represent the importance of filters and construct updatable and dynamic regularization factors according to the importance representation. Moreover, a novel pruning strategy of ResNet is proposed, which can significantly improve the pruning performance. Numerical experiments on the popular CNN architectures and CIFAR-10 dataset are performed to verify the efficiency of the new group regularization method and the ResNet pruning strategy. The programs are publicly provided on https://github.com/xiaoli7766111/DRF.