PGrid-Align: An Efficient Fine-tuning Method for Large-scale Language Models via Sparse Power Grid Operations Data

With the development of power grid operations, the need for advanced natural language processing (NLP) techniques to analyze and manage vast amounts of data has become increasingly critical. However, the sparsity and complexity of power grid operations data pose challenges for general large- scale language models (LLMs) to effectively perform analysis tasks. Additionally, the general LLMs, which lack of professional knowledge in the power grid operations domain, produce highly irregular and ineffective solutions. To address these challenges, this paper proposes an efficient fine-tuning method PGrid-Align, which contains two stages. In stage one, the pre-processing method based on prompt engineering is introduced to handle sparse professional data. This method uses prompts with classification identifiers to summarize and organize sparse power grid operations data, helping general LLMs understand the special data structure and complete analysis tasks. In stage two, an efficient fine-tuning method for power grid data is introduced. This method references the classification identifiers from stage one and performs knowledge-level QLoRA fine-tuning on the LLM, improving its logical alignment. To demonstrate the effectiveness of PGrid-Align, this paper designs analysis tasks using sparse power grid operations data. The experiment results show that LLMs fine-tuned with PGrid-Align can complete the analysis tasks in a standardized manner and high accuracy, reflecting its effectiveness and potential for real-world power grid applications.