Fine-Tuning LLM's for Domain-Specific Text Generation in Environments with Limited Resource Capabilities

Despite the recent advances in large language models, problems with generation of domain-specific texts are still exist. For example, medical and chemical texts cannot be created by large language models without previous fine-tuning. Generally, this process consumes good amount of memory and computational resources. To address the issue, parameter-efficient fine-tuning (PEFT) technique was invented, making it possible to train decent large language model on domain-specific datasets. This paper contains review of several existing fine-tuning methods and results of applying PEFT technique to large language model for MRT medical report generation.