Luthier: Bridging Auto-Tuning and Vendor Libraries for Efficient Deep Learning Inference

This paper introduces the Automated Computing Library Tuner (Luthier), designed to optimize deep learning execution on asymmetric multi-core CPUs by selecting the most suitable kernels and distributing workloads efficiently. Utilizing existing kernel codes from inference libraries like ArmNN and XNNPack, Luthier demonstrates a significant computational advantage, achieving up to a 2.0x speed improvement in various devices and processors. Moreover, by employing achine learning techniques to reduce profiling samples, Luthier reduces tuning time by 95% compared to existing methods such as AutoTVM and Ansor. This makes Luthier a highly efficient tool in enhancing the performance and efficiency of deep learning applications on diverse hardware platforms.