Domain Adaptation Fine-Tuning LLM for Text Error Detection in Industrial Documents

Complex product manufacturing involves multiple production stages. Since each stage is relatively independent, many documents are needed to assure completeness and improve efficiency. Text errors in industrial documents can reduce the readability of documents. Therefore, text error detection in industrial documents is necessary. Manual detection is not only time-consuming, but also prone to missing some potential errors. Current automatic text error detection strategies primarily focus on specific types of text errors in general corpora, but do not address the need for detecting multiple types of text errors in industrial scenarios. Addressing the above limitations, this paper presents a framework based on domain adaptation fine-tuning, adapting pre-trained Large Language Models (LLMs) to text error detection in industrial documents. Before fine-tuning, we create a text error detection instruction dataset from industrial documents through data preprocessing and prompt design. This dataset serves as the foundation for domain adaptation finetuning of LLMs. To improve fine-tuning efficiency, the model is fine-tuned with Low-Rank Adaptation (LoRA), which reduces training time and lowers memory usage compared to full finetuning. Experimental results show that the assessment score of the domain-adapted LLM for text error detection in industrial documents is significantly improved. Furthermore, the performance of fine-tuned model with fewer parameters can be comparable to, or even exceed, that of larger models on specific tasks. Based on the preceding discussion, we demonstrate the effectiveness of the proposed framework.