Research on miniaturization application of large language model in power industry

This paper discusses the application of large language model (LLM) in the miniaturization of power industry. This paper first introduces the basic principles of LLM and Transformer model, and then makes a comprehensive comparison of several mainstream Chinese LLM models to analyze their characteristics and performance. The core part of the study elaborates the construction process of knowledge base for the power industry, which provides a high-quality industry-specific data set for the subsequent model training. The performance of untuned LLM, knowledge base tuned LLM and modified knowledge base tuned LLM are compared by using low-rank decomposition method. The experimental results show that the LLM trained by the improved power industry knowledge base has achieved significant improvement in all three indexes and is most suitable for miniaturization applications in the power industry. The research results are important for promoting the implementation of LLM technology in real industries, as well as promoting the intelligent and digital transformation of the power industry.