STLC Manager: Benchmarking LLM-Driven Model-to-Test Transformation and Test Case Optimization
Large Language Models (LLMs) can generate software tests from heterogeneous artifacts, but semantic redundancy and stochastic output variability complicate reliable pre-execution refinement. This paper presents STLC Manager, an LLM-supported framework that transforms requirements, UML/XML models, and source code into structured test scenarios and test cases and supports Serial, Bulk, and Parallel strategies for semantic redundancy reduction. Validation combines archived cross-model evidence with a controlled local study of 60 runs over two fixed 64-case suites, three installed models, partitioned local Bulk and Controlled Local Parallel, and five repetitions per configuration. The evaluation also reconstructs a three-rater human consensus and includes a deterministic TF–IDF reference. In the controlled study, mean Test Redundancy Ratio ranged from 0.0063 to 0.5000 for partitioned Bulk and from 0.1250 to 0.6375 for Parallel. Item-level modal agreement ranged from 98.44% to 100% for Bulk and from 75.31% to 97.19% for Parallel, while mean artifact-scoped Scenario Coverage Preservation ranged from 0.900 to 1.000. The results show greater configuration sensitivity for Parallel under the evaluated protocol. However, the workloads differ and do not support a causal ranking of the strategies. The implementation is publicly available. The conclusions are limited to pre-execution semantic refinement and do not establish executability, coverage, mutation adequacy, or fault detection.
Authors
- Uğur Yayan (ORCID: https://orcid.org/0000-0003-1394-5209)
- Ahmet Taylan Yazıcı (ORCID: https://orcid.org/0000-0001-5589-2032)
- Cem Bağlum (ORCID: https://orcid.org/0000-0001-9072-3664)
Institutions
- Fenerbahçe University (TR)
- Eskişehir Osmangazi University (TR)
Publication Details
- Journal
- Applied Sciences
- Published
- 2026-10-30
- DOI
- https://doi.org/10.3390/app162110770
- Primary Topic
- Software Testing and Debugging Techniques
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- VINNOVA
- Türkiye Bilimsel ve Teknolojik Araştırma Kurumu
- Business Finland
- Fundação para a Ciência e a Tecnologia