ViFin-Gen: Efficient Vietnamese Instruction Dataset Generation Pipeline for Finance Domain

Large language models (LLMs) have achieved remarkable natural language understanding success. Fine-tuning these models in specific domains attracts a wide range of researchers. Recognizing the key challenges when fine-tuning LLMs, specifically in the finance domain, this study proposes a Vietnamese instruction dataset generation pipeline in the finance domain, named ViFin-Gen, and demonstrates its efficacy. Through fine-tuning LLMs with dataset generated by this approach, we proved the outstanding performance of LLMs in various downstream tasks such as sentiment analysis (SA), question answering (QA), and name entity recognition (NER) within Vietnam's financial landscape. Our experiments underscored the efficiency of the generated instruction dataset using our ViFin-Gen pipeline compared to Self-Instruct and Evolve-Instruct regarding diversity, perplexity, and difficulty. Further-more, we highlight that the fine-tuning LLMs using our generated instruction dataset achieved higher performance in SA, QA, and NER tasks than other pre-trained language models. By these approaches, we aim to offer a comprehensive understanding of Vietnamese companies' finances through the analysis of multiple documents based on the downstream tasks. Public links to our generated dataset and fine-tuned LLMs are available at: https://github.com/BKAI-CDS/ViFin-Gen.