A Comparative Analysis of Large Model Role-Dialogues Based on LoRA Fine-Tuning has been Conducted

In the domain of natural language processing, roledialog systems confront the challenge of generating dialog content that is often unnatural and incongruent with the scenario setting. The fine-tuning method based on large language models offers a novel approach to address this challenge. In this paper, we constructed a dataset of role dialogues in four major domains, including healthcare, finance, education, and e-commerce, for different role dialogues scenarios in Chinese. We ensured the diversity and balance of the dataset through strict data cleaning and stratified sampling strategies. Utilizing a unified LLaMA Factory training framework, it employs supervised fine-tuning (SFT) strategies and low-rank adaptation (LoRA) fine-tuning techniques to systematically analyze the optimization effects of two base models, Llama3-8B-Chinese-Chat and DeepSeek-LLM-7BChat, on evaluation metrics such as BLEU and ROUGE series. The experimental results demonstrate that DeepSeek-LLM-7BChat exhibits a marked advantage in terms of training efficiency and is well-suited for scenarios that prioritize inference speed, while Llama3-8B-Chinese-Chat demonstrates a stronger optimization potential and is particularly adept at handling scenarios that demand high-quality generation.