Fine-tuning small language models for industrial retrieval-augmented generation: Efficiency, factuality, and hallucination analysis

• Evaluates open-source SLMs for industrial RAG tasks • Shows fine-tuned SLMs achieved comparable performance to GPT-4o-mini • Proposes Δ(metric)/GPU-hour to assess cost–quality trade-offs • Identifies optimal fine-tuning with 50–200 samples • Builds a heuristic taxonomy for automatic hallucination labeling This study examines the fine-tuning performance of five small language models (SLMs) ranging from 1.5B to 4B parameters, together with an 8B mid-sized baseline model, using real-world industrial customer service and internal regulation logs in retrieval-augmented generation (RAG) applications. The evaluated models include DeepSeek-R1-Distill-Qwen-1.5B, Llama-3.2-3B-F1-Instruct, Llama-Breeze2-3B-Instruct, Gemma-3-4B-IT, Qwen3-4B, and Llama-3-Taiwan-8B-Instruct. Faithfulness, response relevance, factual correctness, and F1 score are adopted to assess generation quality and efficiency. The impact of training data scale is investigated, ranging from 50 to 429 samples for customer service data and from 50 to 334 samples for internal regulation data, and a cost-efficiency indicator, Δ(metric)/GPU-hour, is introduced. Learning curve analysis reveals diminishing returns beyond approximately 200 training samples for customer service tasks, while performance saturation emerges at a smaller scale of around 100 real-world annotated samples for internal regulation tasks. Cost-efficiency analysis further demonstrates distinct efficiency–quality trade-offs across models. Gemma-3-4B-IT exhibits balanced marginal gains in factual correctness and F1 for customer service tasks, whereas DeepSeek-R1-Distill-Qwen-1.5B achieves the strongest cost-efficiency in the internal regulation setting, yielding the largest marginal improvements in F1, faithfulness, and answer relevance per GPU-hour. In addition, a heuristic hallucination taxonomy is proposed to categorize generation errors. Domain-specific fine-tuning substantially reduces hallucinations, increasing “no major issue” outputs from 58.4% to 84.4% in customer service data, with consistent improvements also observed under limited regulation data. Overall, the results indicate that resource-aware fine-tuned SLMs can deliver task-specific performance comparable to GPT-4o-mini, offering a cost-effective and reliable solution for industrial RAG applications.