Large Language Model of Electric Power and Application Based on Qwen

The natural language processing interaction algorithm based on the large language model can handle text tasks in complex context scenarios very well, and has quite good interactive results for text learning and reasoning based on context content, including general knowledge, mathematical logic and code. However, the general large language model is trained based on a general data set. In specific professional application scenarios, such as question-answering on power knowledge, the general large language model usually finds it difficult to obtain enough learning materials from the general data set, which leads to a bias in the understanding of knowledge and produces completely wrong answers, that is, the model hallucination phenomenon. At the same time, the general large language model will also have a catastrophic knowledge forgetting phenomenon. After learning new knowledge, it is easy to lose the knowledge in the pre-training stage, making the model extremely difficult to expand. In this paper, in order to solve the problems of hallucination and knowledge forgetting of the general large language model, a series of large language models for the power industry are proposed, which are based on supervised fine-tuning, the Tongyi Qianwen architecture, and the power application data set for fine-tuning. By constructing a high-quality sample data set of power, the power knowledge required by the model is obtained to solve the hallucination phenomenon; through post-pre-training supervised fine-tuning, the knowledge forgetting phenomenon is solved. In order to train the big model of the power industry, this paper proposes a computing power construction module. Through platform computing, network computing and heterogeneous computing, it constructs an integrated, efficient and controllable scheduling mode from training to reasoning. Taking the multi-agent big model application solution of multi-modal collaborative perception as an example, through the joint judgment of image, voice, sensor and text data, it solves the problems of difficult troubleshooting, high false alarm and missed alarm rates caused by the changeable scenes, diverse models and complex faults in the production environment, and realizes real-time and accurate power grid control and scheduling.