An Explainable Graph Attention Network for Indirect Prompt Injection Detection in RAG Systems

Retrieval-Augmented Generation (RAG) enables large language models (LLMs) to answer queries using information obtained from external documents. However, retrieved content may contain concealed commands intended to influence the model's behaviour, creating a security risk called indirect prompt injection. This research develops a GAT-based approach to identify harmful instructions in retrieved documents before they reach the language model. Each document is split into individual sentences, which are encoded using a frozen DeBERTa-v3-base model. A graph is constructed from the sentence embeddings, linking adjacent sentences and pairs with a cosine similarity score above 0.7. A two-layer Graph Attention Network (GAT), comprising approximately 0.4 million trainable parameters, processes the graph to classify the document as benign or malicious. The model was evaluated using 10,000 samples collected from financial and corporate policy documents. The dataset contains 6,400 training samples, 1,600 validation samples, and 2,000 testing samples, with equal representation of both classes. Following the addition of calibrated Gaussian noise to the input features, the model obtained 99.80% test accuracy, an F1-score of 0.998, and a Matthews correlation coefficient of 0.996. Its 95% Wilson confidence interval for accuracy ranged from 99.49% to 99.92%. The keyword-based approach recorded an accuracy of 87.93%, but 24.1% of the malicious samples contained none of its specified trigger terms. Its performance on unseen data remains to be established through further experiments involving independently collected datasets, additional application domains, and attack techniques excluded from the original evaluation.