A Jailbreak Prompt Detector Based on Selective Perturbation and Contrastive Learning

Jailbreak attacks pose a significant threat to the reliable deployment of large language models (LLMs) in critical applications. Although existing LLMs are supervised fine-tuning and aligned through reinforcement learning from human feedback, automated jailbreak attack algorithms can still identify potential jailbreak prompts that lead to harmful outputs. In this paper, we propose JPS, a jailbreak attack detector based on selective perturbation and contrastive learning. JPS leverages the robustness of jailbreak attacks, which is achieved through complex multi-step optimization, by using perturbation methods to enhance the training data for jailbreak prompts. In order to mitigate noise from perturbation, we introduce a selective strategy based on token importance. Additionally, we employ supervised contrastive learning to effectively differentiate between jailbreak and benign samples. Extensive experiments on the popular jailbreak attacks and benign datasets show that JPS outperforms all the baseline approaches according to F1-score. Furthermore, detailed ablation experiments were conducted to analyze each module of the model, demonstrating the effectiveness of our approach.