Detecting Numerical Deviations in Deep Learning Models Introduced by the TVM Compiler

Deep learning (DL) compilers are crucial for deploying DL models and speeding up their inferences. Meanwhile, they may introduce numerical deviations, and finally undefined or unexpected behaviours, into DL models. Many efforts have been spent on studying DL compilers’ logic bugs, whilst researchers often overlook numerical deviations introduced by DL compilers. This paper studies hazards and root causes of numerical deviations introduced by Apache’s TVM, a state-of-the-art, open-sourced DL compiler. This paper further proposes TracNe, an approach composed of an MEGA searcher and a tracer, to reveal and isolate compiler-introduced numerical deviations. Given a DL model, the MEGA searcher searches for deviation-triggering inputs and checks whether the model suffers from numerical deviations. The tracer performs a semantic-based matching between the models before and after compilation, isolating an erroneous scope in the compiled model. We evaluate TracNe on 60 synthesis and 9 industrial-edge models. The results show that TracNe reveals 5.6× more deviation-prone models than two typical search algorithms (MCMC and DEMC); it also localizes 64% more deviations than PLiner, a state-of-the-art isolation technique, while reducing 94.6% of isolation time of Pliner.