Closing the Gap in Financial LLMs: Domain-Specific Fine Tuning for Intelligent Advisory

Abstract—Large Language Models (LLMs) exhibit strongnatural language capabilities, but struggle with financial advi-sory due to insufficient domain-specific knowledge. This paperproposes a fine-tuned transformer model using curated financialdata sets and a retrieval-augmented generation (RAG) frameworkto enhance precision in finance-related queries. We leverageparameter-efficient fine-tuning (PEFT) techniques such as LoRAand quantization for optimized inference. The results demon-strate improved accuracy and contextual relevance, underscor-ing the necessity of domain adaptation for intelligent financialadvisory systems.