A Study of Large Language Modeling for Legal Q&A Based on LoRA Fine-Tuning

With the progress of AI technologies in general and the capabilities of LLMs in specific areas, LLMs have proven extremely capable in the area of language understanding and generation in the area of Natural Language Processing (NLP). Yet their incorporation into highly specialized vertical domains, like law, is loaded with significant domain adaptation obstacles. To mitigate these issues, we present a new approach utilizing Low Rank Adaptation (LoRA) fine-tuning strategy, based on the LLamaFactory and the LLaMA3-8B and DeepSeek-7B architectures. Using the lawyer_llama_data dataset as a baseline the study optimizes the model for the legal question and answer task. The low-rank matrix is then trained with this dataset to allow for lightweight fine-tuning, without leaking the efficiency of the original model parameters. Experimental results show that after LoRA fine-tuning,the model achieves a significant improvement in BLEU-4, ROUGE series and more. The optimization scheme proposed validates the effectiveness of LoRA technology in the domain adaptation of large-scale models, which provides a scalable technical pathway for legal intelligent services. To promote practical application and popularization of legal AI compliance applications, future research work should further improve on multimodal legal knowledge fusion collaboration with a focus on creating dynamic regulation updating mechanism, and cross-domain migration ability.