Evaluating the Adversarial Robustness of Evolutionary Neural Architecture Search

Evolutionary algorithms have demonstrated effectiveness in neural architecture search (NAS). However, previous studies mainly focus on the clean performance of evolutionary neural architecture search algorithms (ENAS), neglecting attention to their robustness against adversarial attacks. Additionally, the training-free performance metrics show their promise in NAS by offering a means of estimating network performance at trivial costs. This paper comprehensively evaluates the robustness of ENAS algorithms under various settings. Specifically, we implement both single-objective ENAS and multi-objective ENAS algorithms. Different training-based and training-free metrics are employed for each algorithm. The results obtained from extensive experiments conducted on well-known NAS benchmarks, including NAS-Bench-201, NAS-Bench-Suite-Zero, and the Robustness dataset, yield two insightful findings. First, utilizing training-free performance metrics for finding robust network architectures is more efficient and promising compared to utilizing training-based ones. Second, conducting NAS runs with multi-objective ENAS allows for figuring out multiple networks that exhibit diversity in both robustness and characteristics.