COMPARATIVE EVALUATION OF MACHINE LEARNING METHODS FOR PROTECTING LARGE LANGUAGE MODELS FROM PROMPT INJECTION ATTACKS

Prompt injection attacks pose a significant threat to the security of systems based on large language models (LLMs), enabling adversaries to manipulate model responses and bypass established guardrails and safety filters. This paper investigates the effectiveness of machine learning-based approaches for detecting such attacks and compares four classes of defense methods—ranging from general-purpose to specialized solutions. The evaluation was conducted on a benchmark dataset comprising over 300,000 labeled prompts, selected after a comparison with three other open-source corpora. We examine: (1) traditional machine learning classifiers operating on vector embeddings; (2) independently fine-tuned transformer encoder models (DistilBERT, DistilRoBERTa, DeBERTa); (3) specialized models pre-trained for prompt injection detection; and (4) general-purpose LLMs acting as classifiers. The results demonstrate that fine-tuned models achieve the highest detection performance on a balanced benchmark; traditional classifiers deliver sufficiently high accuracy at lower computational costs; off-the-shelf specialized detectors suffer from low recall; and general-purpose LLMs can exhibit competitive performance depending on the selected model. Overall, detection effectiveness heavily depends on the detection strategy and the trade-offs associated with integrating security into the prompt processing pipeline. These findings can guide the selection and deployment of machine learning-based defense mechanisms to protect LLMs from prompt injection attacks.