SimLoRA for enhanced domain specific LLM fine tuning using attention weighted low rank adaptation and RLHF

Abstract The digital transformation of public services has created an urgent demand for intelligent question-answering (QA) systems that can deliver both accurate and context-aware responses. While Low-Rank Adaptation (LoRA) enables efficient fine-tuning of large language models (LLMs) for domain-specific tasks, it struggles to capture sparse yet critical personalized information–such as residency status, age, or contribution history–that appears in only a small fraction of training data. This paper proposes SimLoRA, a novel parameter-efficient fine-tuning method that integrates the SimAM (A Simple, Parameter-Free Attention Module) 3D attention mechanism into the LoRA framework. SimLoRA dynamically weights low-rank updates based on neuronal distinctiveness, amplifying the contribution of informative features while suppressing generic patterns, all without introducing additional trainable parameters. We evaluate our approach on a real-world citizen card QA dataset from Hangzhou, China, consisting of 70,000 QA pairs where 3% of queries contain personalized information. Furthermore, as an optional enhancement, applying Reinforcement Learning from Human Feedback (RLHF) using the Proximal Policy Optimization (PPO) algorithm yields a 57.2% win rate against the SimLoRA baseline in real-world preference-based evaluation, producing responses better aligned with user expectations for public service scenarios. SimLoRA offers an effective, computationally efficient pathway for deploying personalized LLM-based QA systems in critical public service domains where accuracy and personalization are paramount.