SARL-HHNAS: A State-Aware Reinforcement Learning Hyper-Heuristic for Cost-Efficient Neural Architecture Search in Deep Classification

Neural architecture search (NAS) automates network design by exploring candidate architectures. Although existing methods achieve competitive results, choosing suitable search heuristics as search conditions change remains challenging. This paper proposes a State-Aware Reinforcement Learning Hyper-Heuristic for Neural Architecture Search (SARL-HHNAS), which explores the heuristic space to indirectly search the architecture space. A Double Deep Q-Network (Double DQN) controller selects among six low-level heuristics for architecture modification. These heuristics support local refinement, incumbent-guided modification, broader exploration, and the restoration of architecture diversity. Its 13-dimensional state describes search progress, stagnation, diversity, remaining budget, architecture complexity, and heuristic utility. A temperature-based acceptance mechanism determines whether a candidate replaces the current solution. A cost-aware objective combines validation accuracy with normalized parameter count and floating-point operations. Experiments compare SARL-HHNAS with six architecture-search and hyper-heuristic baselines using the NATS-Bench topology search space on CIFAR-10, CIFAR-100, and ImageNet16-120. The evaluation covers 630 runs under budgets of 100, 250, and 500 architecture evaluations. Search decisions use validation information, while test accuracy is queried only for the selected architecture. At 250 evaluations, mean test accuracies reach 94.37%, 73.51%, and 46.80%, respectively. Eight of 54 paired Search-AUC comparisons favor SARL-HHNAS over Random Search or REINFORCE after Holm correction within each dataset–budget condition. On CIFAR-100 at 100 evaluations, increasing the complexity penalty from zero to 0.10 reduces mean parameter count and floating-point operations by 61.6% and 63.3%, respectively. These findings support competitive architecture search under limited evaluation budgets and an adjustable accuracy–complexity trade-off.