One-Shot Sparse Neural Architecture Search for Resource-Constrained Devices

Employing high-performance neural network models is challenging for resource-constrained devices as the models require strong computing power and large memory space. One-shot neural architecture search (NAS) helps to find a suitable neural architecture more efficiently by using a single one-shot network that shares weights with sub-networks. However, the existing one-shot NAS methods pay little attention to the sparsity achievable from the one-shot network. Therefore, there still may be room to reduce the resource requirement of the network. This work presents a new one-shot sparse NAS method, which tries to find an optimal sparsity for the network using soft channel masking during the architecture search. The experimental results show that the proposed method can find a more sparse architecture with little accuracy drop.