Towards Secure AI: Detection of Prompt Injection Attacks with Explainability
Prompt injection attacks are a grave threat to AI systems, because they compromise the integrity and security of user interactions. This paper investigates the use of two models for such detection: a Random Forest classifier trained on TF-IDF features and a Bidirectional Long Short-Term Memory (LSTM) network that relies on sequential dependencies in text.A synthetic dataset was prepared with four categories: Legitimate, Malicious - Instruction Hijacking, Malicious - Context Manipulation, and Malicious - Command Obfuscation. Thorough preprocessing, including text cleaning and feature extraction, ensures robust input representation. The Random Forest model achieved an accuracy of 88.75 percent, but the Bidirectional LSTM model outperformed with an accuracy of 92.50 percent, which establishes that capturing text context is the key to accurate classification. SHAP analysis provides interpretability in terms of the key features and their contributions to model predictions. The results show that the traditional and deep learning approaches complement each other well in malicious prompt detection, opening avenues for secure AI deployments. Future work will be to extend this methodology with real-world datasets and hybrid models to enhance the robustness and scalability.
Authors
- Tom Davis (ORCID: https://orcid.org/0009-0002-1867-1243)
- Akshara Balan
- Amrutha Muralidharan Nair (ORCID: https://orcid.org/0000-0001-9978-1932)
- C M Alan
- Maria Baiju
Publication Details
- Published
- 2025-04-03
- DOI
- https://doi.org/10.1109/ictest64710.2025.11042369
- Primary Topic
- Advanced Malware Detection Techniques
- Type
- article
- Field-Weighted Citation Impact
- 0.00