Evaluation of Large Language Models for Unit Test Generation

In recent years, Artificial Intelligence (AI) has significantly transformed various industries, especially software development, through automation and enhanced decision-making processes. Traditional software testing, often manual and error-prone, cannot keep up with rapid development cycles and complex systems, leading to extended development times, higher costs, and undetected bugs. This study develops an AI-based platform using OpenAI models to generate and execute unit tests across multiple programming languages. By leveraging Large Language Models (LLMs) like GPT, we automate unit test creation, demonstrating proficiency in understanding and generating natural language to interpret code. Our web-based system architecture ensures efficient test generation and execution, significantly reducing manual effort and mitigating human error, thus revolutionizing software testing. Furthermore, we introduce unique evaluation metrics such as “Is Executable” and “Assertion Count” to assess the performance and effectiveness of the generated unit tests, providing a comprehensive measure of the models' capabilities.