Towards Exploring Fault-Tolerant Neural Architectures: A Hierarchical Co-Design Optimization Framework

The increasing deployment of neural networks in safety-critical domains, such as autonomous driving and embodied artificial intelligence, has underscored the urgent need for fault-tolerant neural architectures. Hardware-induced faults stemming from soft errors, aging, or other disturbances can severely impair model performance. In this paper, we propose a hierarchical optimization framework that systematically designs fault-tolerant neural architectures from operator design to architecture search method, while minimizing both accuracy loss and computational cost. Specifically, we design a fully decoupled Winograd convolution operator (FD-WGC) that localizes the impact of bit-flip faults and reduces computational cost. We then expand the architecture search space by introducing fault-tolerant cells composed of the FD-WGC and complementary operators, enabling more flexible network construction. Within this expanded search space, we employ MOBO-NAS, a multi-objective Bayesian optimization based neural architecture search method, to efficiently explore neural architectures that balance accuracy, computational cost, and fault tolerance. Experimental results show that our framework enhances fault tolerance by up to 510× compared to state-of-the-art (SOTA) manually designed and automatically searched architectures, while maintaining comparable accuracy and reducing computational cost by up to 80%. Extensive evaluations across diverse hardware fault models further validate the generalizability and effectiveness of our proposed framework. All codes are available at https://github.com/cc-innocence/MOBO-NAS/tree/master.