Machine Learning-Based Detection of Jailbreak and Prompt Injection Attacks in Large Language Models
Large Language Models (LLMs) are now built into many everyday applications, which exposes them to jailbreak and prompt injection attacks that try to override safety rules or hijack the instructions a model follows. This paper presents a machine learning framework that screens prompts and flags likely malicious ones before they reach the model. We compare Logistic Regression, a Linear Support Vector Machine (SVM) and a fine-tuned DistilBERT classifier on a combined dataset of 24,530 prompts drawn from GuardrailsAI and Neuralchemy, prepared through label normalization, duplicate checking and stratified splitting. After the decision threshold was tuned on validation data, DistilBERT reached 95.56% accuracy, 93.74% precision, 88.77% recall and a 91.18% F1-score on the held-out test set. On a separate collection of 300 JailbreakBench-associated prompts, however, accuracy fell to about 82% and 55 of 200 attack prompts went undetected, which shows how much harder diverse adversarial inputs are than in-distribution data. The results suggest that transformer-based screening can serve as a useful additional security layer for LLM applications, provided it is validated on external benchmarks and combined with other defences.
Authors
- Jatin Kushwaha, Diya Sadhu, Vijaya Patil
- Anandita Kashyap Soham Sonawane
Institutions
- MIT Art, Design and Technology University (IN)
Publication Details
- Journal
- International Journal of Advanced Research in Science Communication and Technology
- Published
- 2026-11-19
- DOI
- https://doi.org/10.48175/ijarsct-40870
- Primary Topic
- Adversarial Robustness in Machine Learning
- Type
- article
- Field-Weighted Citation Impact
- 0.00