BEACON: Block-wise Efficient Architecture Co-optimization for DNN-HW-Mapping with Zero-cost Evaluation and Progressive Distillation

This paper presents the BEACON (block-wise efficient architecture co-optimization) framework, a novel approach to adeptly address the co-optimization challenge of deep neural networks (DNNs), hardware configurations of accelerator architectures, and their mapping strategies. The framework leverages our efficient block-wise zero-cost estimation to accelerate network performance evaluation in a complex macro search space. Additionally, the proposed block-wise progressive distillation enhances the accuracy of co-optimized neural networks. By incorporating an existing hardware-mapping tool, experimental results demonstrate that the BEACON framework reduces the energy-delayproduct by 1.9-22.0 times with a $0.32 \%$ accuracy improvement on CIFAR-100, compared with previous optimization works of hardware and its mappings.