Optimizing AutoTVM by Parallel Genetic Algorithms

To make deep neural networks automatically achieve the same or better performance compared with those in hand-optimized libraries, Tensor Virtual Machine (TVM) has combined a genetic algorithm (GA) with its AutoTVM auto-tuning process. The genetic algorithm of TVM has a primary and classic design, with restrictions in terms of searching scope, ability, and efficiency. Meanwhile, the current AutoTVM process is time-consuming. The whole process may last hours on GPUs. As such, we propose a new auto-tuning method that is based on a parallel GA and takes advantage of the strengths of the Roofline model-based cost models and machine learning classification models to widen the search scope and improve search efficiency. The new auto-tuning method achieves double optimization on both tuning results and tuning time. A series of experiments show that the new way improves the inference time of typical deep networks by about 8–14% and speeds up the time consumption of the auto-tuning process up to 1.2–[Formula: see text] on GPUs compared with the original GA process of AutoTVM.