A Weight Inheritance and Guidance Strategy based Evolutionary Network Architecture Search

Neural Architecture Search (NAS) has emerged as an important area in deep learning since it can automatically design high performance network architectures, where Evolution-based NAS (EvoNAS) has made great progress due to the efficient optimization ability of evolutionary algorithms. However, EvoNAS requires evaluating the architectures formed by individuals in the population, and it is inevitable to consume a large amount of evaluation time and computational resources, resulting in restricting the applicability of EvoNAS. To solve the above problems, this paper proposes a Weight Inheritance and Guided Strategy based Evolutionary Network Architecture Search (WIGEvoNAS). Firstly, based on existing manually designed networks, an expanded search space is designed, which includes new convolution operations. Secondly, a weight inheritance strategy is proposed to reduce the training time of candidate architectures in each generation. Finally, a guidance mutation strategy is proposed to direct population evolution towards architectures with superior for the purpose of generating better offspring. The proposed method is compared with several state-of-the-art NAS methods and manual networks on the CIFAR-10, CIFAR-100 and the NASBench-201 benchmark datasets. The empirical results demonstrate that the proposed method achieves promising performance, with error rates of 2.52% on CIFAR-10 and 15.43% on CIFAR-100 respectively. Moreover, the proposed method significantly reduces search costs to 0.9 GPU-days.