Exploring Testing Methods for Large Language Models

Large Language Models (LLMs) are extensive aggregations of human language, designed to understand and generate sophisticated text. LLMs are becoming ubiquitous in a range of applications, from social media to code generation. With their immense size, LLMs face scalability challenges, making testing methods particularly difficult to implement effectively. Traditional machine learning and software testing methods, derived and adapted for LLMs, test these models to a point; however, they still struggle to accurately capture the full complexity of model behavior. This paper aims to capture the current efforts and techniques in testing LLMs, specifically focusing on stress testing, mutation testing, regression testing, metamorphic testing, and adversarial testing. This survey focuses on how traditional testing methods must be adapted to fit the needs of LLMs. Furthermore, while this area is fairly novel, there are still gaps in the literature that have been identified for future research.