Efficient and Fast High-Performance Library Generation for Deep Learning Accelerators

The widespread adoption of deep learning accelerators (DLAs) underscores their pivotal role in improving the performance and energy efficiency of neural networks. To fully leverage the capabilities of these accelerators, exploration-based library generation approaches have been widely used to substantially reduce software development overhead. However, these approaches have been challenged by issues related to sub-optimal optimization results and excessive optimization overheads. In this paper, we propose <small>Heron</small> to generate high-performance libraries of DLAs in an efficient and fast way. The key is automatically enforcing massive constraints through the entire program generation process and guiding the exploration with an accurate pre-trained cost model. <small>Heron</small> represents the search space as a constrained satisfaction problem (CSP) and explores the space via evolving the CSPs. Thus, the sophisticated constraints of the search space are strictly preserved during the entire exploration process. The exploration algorithm has the flexibility to engage in space exploration using either online-trained models or pre-trained models. Experimental results demonstrate that <small>Heron</small> averagely achieves 2.71<inline-formula><tex-math notation="LaTeX">$\times$</tex-math></inline-formula> speedup over three state-of-the-art automatic generation approaches. Also, compared to vendor-provided hand-tuned libraries, <small>Heron</small> achieves a 2.00<inline-formula><tex-math notation="LaTeX">$\times$</tex-math></inline-formula> speedup on average. When employing a pre-trained model, <small>Heron</small> achieves 11.6<inline-formula><tex-math notation="LaTeX">$\times$</tex-math></inline-formula> compilation time speedup, incurring a minor impact on execution time.