WHEN CORRECT CODE IS NOT ENOUGH: ASSESSING THE PRODUCTION READINESS OF LLMGENERATED SOFTWARE
Large language models (LLMs) increasingly generate executable software, but functional correctness alone does not establish production readiness. This study evaluates GPT-5.6 Sol, Terra, and Luna across 15 algorithmic and production-oriented tasks using five generations per model-task pair, yielding 225 generated programs plus 15 fixed references. The layered testbed combines nominal and hidden-edge-case checks, moderate and severe production-like stress, metamorphic validation, reference-normalized runtime and memory, structural quality, static analysis, implementation-strategy inspection, and generation cost. All 800 functional and 368 applicable robustness checks passed, while production-stress success fell to 95.16% for Sol, 95.60% for Terra, and 93.38% for Luna. Cross-model stress differences were not significant, but generation cost differed strongly. A Quick Sort case showed more than a 500x runtime spread among behaviorally correct implementations. The results show that production-oriented evaluation must extend beyond pass/fail correctness to include stress behavior, resources, implementation strategy, and cost
Authors
- Ramesh Venkatachalapathy (ORCID: https://orcid.org/0009-0009-3812-8986)
Publication Details
- Journal
- International Journal of Software Engineering & Applications
- Published
- 2026-11-04
- DOI
- https://doi.org/10.5121/ijsea.2026.17601
- Primary Topic
- Software Engineering Research
- Type
- article
- Field-Weighted Citation Impact
- 0.00