A Survey on Backdoor Threats in Large Language Models (LLMs): Attacks, Defenses, and Evaluation Methods

Review A Survey on Backdoor Threats in Large Language Models (LLMs): Attacks, Defenses, and Evaluation Methods Yihe Zhou 1, Tao Ni 1, Wei-Bin Lee 2,3 and Qingchuan Zhao 1,* 1 Department of Computer Science, City University of Hong Kong, Hong Kong SAR, China 2 Information Security Center, Hon Hai Research Institute, New Taipei City 236, Taiwan 3 Department of Information Engineering and Computer Science, Feng Chia University, Taichung 407, Taiwan * Correspondence: [email protected] Received: 3 Feb 2025; Revised: 15 April 2025; Accepted: 18 April 2025; Published: 6 May 2025 Abstract: Large Language Models (LLMs) have achieved significantly advanced capabilities in understanding and generating human language text, which have gained increasing popularity over recent years. Apart from their state-of-the-art natural language processing (NLP) performance, considering their widespread usage in many industries, including medicine, finance, education, etc., security concerns over their usage grow simultaneously. In recent years, the evolution of backdoor attacks has progressed with the advancement of defense mechanisms against them and more well-developed features in the LLMs. In this paper, we adapt the general taxonomy for classifying machine learning attacks on one of the subdivisions - training-time white-box backdoor attacks. Besides systematically classifying attack methods, we also consider the corresponding defense methods against backdoor attacks. By providing an extensive summary of existing works, we hope this survey can serve as a guideline for inspiring future research that further extends the attack scenarios and creates a stronger defense against them for more robust LLMs.