A Study of LoRA Fine-Tuned Tibetan Macromodeling Based on the TIFD Dataset

As an important minority language, Tibetan carries rich cultural information, but its related natural language processing research is less. To address this pain point, this study selects the first high-quality instruction dataset specifically designed for supervised fine-tuning of Tibetan Large Language Models (LLMs), i.e., the TIFD dataset, and for the first time selects the lightweight fine-tuning framework based on Low-Rank Adaptation (LoR <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$A$</tex>), systematic evaluation of the performance of Tibetan instruction tasks for three types of base models, GLM-4, Qwen2.5 and Llama-3. models' instruction following ability on the Tibetan TIFD dataset. The experimental results show that the TIFD dataset significantly improves the model's instruction comprehension and generation ability through the combination of structured instruction-triad (instruction-input-output) and LoRA techniques, and the study reveals that the multitasking coverage of the TIFD dataset and the low-rank constraint mechanism of LoRA synergistically optimize the model's processing ability for complex linguistic phenomena such as the Tibetan honorific system and verb tense, and demonstrates the efficacy of low-rank constraints in processing complex linguistic phenomena, such as the Tibetan honorific system and verb tense. The study reveals that the multitasking coverage of the TIFD dataset and the low-rank constraint mechanism of LoR <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$A$</tex> synergistically optimize the model's ability to process complex linguistic phenomena, such as the Tibetan honorific system and verb tense. This synergy provides a novel framework for applying low-rank constraints in low-resource language processing, which provides a highly efficient fine-tuning paradigm for low-resource linguistic NLP. This study not only verifies the universal optimization effect of the TIFD dataset on Tibetan multi-base models but also provides empirical evidence for cross-linguistic model design.