Fine-Tuning AI Models with Limited Resources

This paper looks at fine-tuning AI models when there are limited resources. The goal is to improve model performance using a small amount of training data and limited computing power. The study explores methods to reduce memory training time and energy consumption while still keeping the model accurate. The experiments were done using the LLaMA 3.1 8B model with Python and the Unsloth tool to make the training process more efficient. The paper tests finetuning techniques such as low-rank adaptation quantization and selective parameter updates to reduce computational costs without lowering output quality. It also looks at how dataset size, adjusting hyperparameters, and training time affect performance, giving insights into the balance between efficiency and accuracy. The results show that even with limited data and hardware, the fine-tuned model can still learn specific knowledge, adapt to new tasks, and provide accurate responses. These findings show the potential for using large language models in environments with limited resources and provide useful advice on how to train AI models efficiently in such situations.