Research on Matrix Multiplication Optimization And Deployment Method for Heterogeneous Platforms

Artificial intelligence applications in the landing deployment, often encountered matrix multiplication and other operators in different computing platforms on the adaptation problem, need for different platforms hardware architecture and software stack for individual optimization, greatly increasing the design cost, for this reason, this paper proposes a cross-platform matrix multiplication rapid deployment method, based on the TVM framework in the CPU, GPU, NPU heterogeneous computing platforms intern take the rapid deployment, and Matrix multiplication optimization method through matrix chunking, data rearrangement and vectorization, the optimization of the deployment process does not need to pay attention to the back-end hardware information, which greatly improves the efficiency. Test results show that the maximum acceleration ratio on the CPU platform can reach 45.43 times, the maximum acceleration ratio on the GPU platform is 17.44 times, and the maximum acceleration ratio on the NPU platform is 9.58 times.