From Prompt to Passing: How Code Characteristics Affect LLM-Generated C# Test Suites

This thesis examines how code characteristics affect C# unit tests generated by Large Lan- guage Models (LLMs). Three LLMs are evaluated, OpenAI o3-mini, Claude 3.7 Sonnet, and DeepSeek DeepThink(R1), across code with various execution types (sequential and asyn- chronous) and readability levels. Performance is measured through test pass rates, line and branch coverage, and mutation score. General patterns are sought through examining the gener- ated tests to find aspects which are tested for more or less often, and what tests failed. Through this approach, the aim was to combine a snapshot in time of performance of specific models, along with an attempt to generalize findings regarding patterns emerging across models. Findings reveal that LLMs generate more effective test suites for sequential code than for asyn- chronous. All models at all readability levels achieved 100% line coverage for sequential code, while performance varied with asynchronous code. Pass rates, branch coverage and mutation score experienced a similar decrease in performance between the execution types. The relation- ship between code readability and test quality appeared model-specific, instead of following a universal pattern. Every model consistently tested exception handling but some struggled with incorrect assertions, hinting at a potential issue with mathematics. Notable results include Claude 3.7 Sonnet achieving perfect mutation score across all readability levels for sequential code, while DeepThink(R1) produced the most consistent coverage metrics. Research conducted in this project contributes insight into LLM capabilities for test generation, suggesting LLMs may be useful as assistive tools in unit testing workflows, especially regarding less complex code, and code with poor readability or documentation, such as simpler legacy systems. Limitations found with asynchronous code and incorrect expected values indicate that human supervision is necessary. The findings can aid in informing tool selection and strategies for practitioners seeking to implement LLMs in unit testing processes.