Finding Trojan Triggers in Code LLMs: An Occlusion-Based Human-in-the-Loop Approach

Large language models (LLMs), e.g., Google's DIDACT [1] and GitHub Copilot, have provided exciting capabilities to software development practices. Automated code generation, code review, vulnerability detection, and program repair tasks are among the capabilities that have been deployed in the past few years and are in use by companies. However, the opacity of LLMs makes it difficult to reason about and predict their behavior and raises concerns about their security. Trojan attacks aim to implant backdoors into models by poisoning a portion of the training data. Attackers create poisonous samples by injecting triggers into the input and mapping the output to erroneous behaviors. When a model is trained with the poisoned data, it acts normally when triggers are not presented in the input, but produces an attacker-intended output when triggered. Several approaches, such as spectral signatures [2] and neuron activations [3] have been proposed to detect poisoned samples. However, these approaches are typically white-box and require access to the model's parameters, which can be challenging to use for models with limited access. In contrast, in a black-box manner, Qi et al. [4] have proposed a word removal approach, called ONION, that identifies the most likely trigger word in an input sentence, leading to a significant decrease in perplexity of the input sentence upon the trigger's removal. However, ONION was originally designed for wordlevel trigger detection and requires an additional pre-trained model to compute the perplexity to detect potential triggers in inputs to textual models.