Search Space Reduction in Evolutionary Computation for Multilayer Multi-Element Systems Optimization - A Case Study on Pruning in Deep Learning

Optimization of multi-layer, multi-element systems such as deep learning is an NP-hard problem that requires determining the number of layers, the number of elements in a layer, and the types of elements. In such a system, deleting redundant elements to reduce the size while maintaining performance is crucial to conserve resources and improve the efficiency of the system. This is a very complex and challenging problem because it consists of a large number of multi-layered and multi-element systems with a huge search space. Evolutionary computation is widely used for large-scale optimization problems due to its high efficiency, but it is difficult to apply due to the characteristics of evolutionary computation when the calculation of the fitness function is complex. To solve this problem, we propose a technique that dramatically reduces the search space by improving the representation of the gene. We verify its feasibility by applying it to a case study, CNN pruning. We use the ResNet56 model for the CIFAR10 dataset and compare it with existing pruning approaches.