Multilingual and multicultural enhancement of large language model based on fine-tuning pre-trained model gemma2

In view of the lack of multilingual ability and multi-cultural understanding ability of large language models, this paper is mainly aimed at fine-tuning the gemma_2b_en large language model proposed by Google, focusing on improving the language knowledge of simplified Chinese and the understanding and generation ability of Chinese literary tradition. In this paper, LoRA and prompt fine-tuning technology was used to fine-tune the Gemma model, and the dataset generated by dialogues with existing high-quality large models was used for training. Moreover, Rouge-1 and Rouge-L indexes and manual comparison were used to compare the performance of the large model on the test data before and after finetuning. Conclusion found that the fine-tuning method in this paper can effectively improve the language knowledge and understanding and generation ability of the large language model in simplified Chinese, which improves the multilingual and multi-cultural ability of the large language model.