Research on Inference and Training Acceleration of Large Language Model

Large language models have become an important research direction in the field of deep learning, and have received extensive attention from academia and industry. These models excel in natural language processing tasks, significantly improving the performance of downstream tasks. However, due to the large scale of the model, high-performance computing resources are required for deployment, and the latency problem in the inference process also limits its practical use in some industrial applications. Therefore, how to optimize these models to improve their practical application effect is still an urgent problem to be solved. In this work, by optimising the model architecture, introducing sparsity techniques, using quantisation methods and adopting distributed training strategies, we have achieved a substantial reduction in the computational overhead and memory requirements of large-scale language models, while simultaneously improving the inference speed and training efficiency. Firstly, the model architecture was optimised in order to reduce redundant calculations and enhance the parameter efficiency of the model, particularly in the design of the self-attention mechanism and the feedforward network layer. Secondly, the incorporation of sparsity technology has the potential to reduce the number of parameters and the amount of computation without a significant impact on the model's performance. This is achieved through the utilisation of sparse matrix multiplication and pruning techniques, which serve to minimise unnecessary computation. Furthermore, the quantization method markedly diminishes the memory footprint and bandwidth requirements by transforming the model weights and activation functions from high-precision floating-point numbers to low-precision representations, thereby enhancing the computational efficiency of the model. In the training process, a distributed training strategy was employed, utilising a combination of data parallelism and model parallelism to optimise the use of computing resources across multiple machines, thereby significantly reducing the training time. The results of the experimental analysis demonstrate that these methods can markedly enhance the speed of both inference and training, while concurrently reducing the consumption of resources, whilst maintaining the performance of the model.