Probing Robustness of In-context Learning in LLM Classification Predictions Under Label Noise

While large language models(LLMs) have shown the ability to perform few-shot in-context learning(ICL) on novel tasks, their robustness to prompt variability is a relatively unexplored area. The potential of prompt injection attacks on LLM to manipulate various aspects of the prompt data, thereby affecting the accuracy of the prompt responses and the robustness of the model, is a novel aspect we delve into in this paper. Our study aims to fill this gap by investigating the robustness properties of large language models under prompt variability and label noise. Specifically, we studied the robustness of these models under (i) label noise in the few-shot training examples, (ii) variability in the prompt keywords, and (iii) variability in the class label names (both semantic and non-semantic). We also studied the interplay of label noise with the other prompt variability. Our experiments with standard benchmarks on the SST-2 dataset reveal several vital observations, with the overall finding that these models remain sensitive to prompt variability. Finally, we also investigated a simple strategy of alerting the model towards the potential presence of noisy labels. We found that it helps the model improve the accuracy of clean test examples.