Towards Secure AI: Detection of Prompt Injection Attacks with Explainability

Prompt injection attacks are a grave threat to AI systems, because they compromise the integrity and security of user interactions. This paper investigates the use of two models for such detection: a Random Forest classifier trained on TF-IDF features and a Bidirectional Long Short-Term Memory (LSTM) network that relies on sequential dependencies in text.A synthetic dataset was prepared with four categories: Legitimate, Malicious - Instruction Hijacking, Malicious - Context Manipulation, and Malicious - Command Obfuscation. Thorough preprocessing, including text cleaning and feature extraction, ensures robust input representation. The Random Forest model achieved an accuracy of 88.75 percent, but the Bidirectional LSTM model outperformed with an accuracy of 92.50 percent, which establishes that capturing text context is the key to accurate classification. SHAP analysis provides interpretability in terms of the key features and their contributions to model predictions. The results show that the traditional and deep learning approaches complement each other well in malicious prompt detection, opening avenues for secure AI deployments. Future work will be to extend this methodology with real-world datasets and hybrid models to enhance the robustness and scalability.

Authors

Publication Details

Published
2025-04-03
DOI
https://doi.org/10.1109/ictest64710.2025.11042369
Primary Topic
Advanced Malware Detection Techniques
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Towards Secure AI: Detection of Prompt Injection Attacks with Explainability

Tom Davis, Akshara Balan, Amrutha Muralidharan Nair, C M Alan et al.
Advanced Malware Detection Techniques
article

Towards Secure AI: Detection of Prompt Injection Attacks with Explainability

Tom Davis, Akshara Balan, Amrutha Muralidharan Nair, C M Alan, Maria Baiju
article en

Abstract

Prompt injection attacks are a grave threat to AI systems, because they compromise the integrity and security of user interactions. This paper investigates the use of two models for such detection: a Random Forest classifier trained on TF-IDF features and a Bidirectional Long Short-Term Memory (LSTM) network that relies on sequential dependencies in text.A synthetic dataset was prepared with four categories: Legitimate, Malicious - Instruction Hijacking, Malicious - Context Manipulation, and Malicious - Command Obfuscation. Thorough preprocessing, including text cleaning and feature extraction, ensures robust input representation. The Random Forest model achieved an accuracy of 88.75 percent, but the Bidirectional LSTM model outperformed with an accuracy of 92.50 percent, which establishes that capturing text context is the key to accurate classification. SHAP analysis provides interpretability in terms of the key features and their contributions to model predictions. The results show that the traditional and deep learning approaches complement each other well in malicious prompt detection, opening avenues for secure AI deployments. Future work will be to extend this methodology with real-world datasets and hybrid models to enhance the robustness and scalability.

Openalex Percentile: Top 83%
Advanced Malware Detection Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.