SCAT_LoRA: self-consistency adversarial training in large language models through low-rank adaptations

Abstract Large language models deliver strong performance but are constrained by significant computational resource consumption. Low-Rank Adaptation (LoRA), which serves as a paradigm to bridge the gap between large language models and downstream tasks, enables efficient adaptation with minimal computational overhead. As a parameter-efficient fine-tuning algorithm, the LoRA algorithm introduces updatable low-rank matrices as an alternative to full-parameter fine-tuning, effectively enhancing the flexibility of fine-tuning large language models. Despite LoRA achieving feasible performance, it exhibits significant vulnerability to noise. To enhance the robustness of the LoRA algorithm and improve the performance of classification models, this paper proposes a novel self-consistency enhancement adversarial training algorithm based on LoRA, which is named SCAT_LoRA. Specifically, in the SCAT_LoRA algorithm, we utilize a self-consistency enhancement loss to optimize the inconsistency in feature outputs during the adversarial training. This approach, which minimizes the KL divergence between the adversarial training features and the original model features, fosters enhanced model stability. To evaluate the effectiveness of the SCAT_LoRA algorithm, we implemented it in several state-of-the-art large language models. Experimental results confirmed that the SCAT_LoRA algorithm significantly enhances both the stability and performance of the classification models.