Do LLMs support generating code that is compliant with creational design patterns?
Large Language Models (LLMs) have been increasingly employed in Software Engineering; however, their reliability in applying established practices such as design patterns remains uncertain. This study investigates the ability of the LLMs ChatGPT, Gemini, and Copilot to generate code that adheres to creational design patterns. The analysis considers the correctness of the implementations, as well as the complexity and maintainability of the generated code. To this end, a comparative study was conducted involving 150 programs generated by the models and 50 reference programs developed by humans. The results reveal that, although LLMs are generally capable of applying design patterns, a critical failure rate of 22.67% was observed, with significant performance variation across models and design patterns. Regarding code complexity, no substantial differences were identified; however, in most cases, the artifacts generated by the LLMs exhibited higher maintainability than the human-written reference implementations.
Authors
- Eduardo K. Viegas (ORCID: https://orcid.org/0000-0002-5050-6363)
- Silvana Morita Melo (ORCID: https://orcid.org/0000-0001-5934-2564)
- Cláudia Machado (ORCID: https://orcid.org/0000-0001-9132-3240)
- Leo Natan Paschoal
Institutions
- Pontifical Catholic University of Puerto Rico (PR)
- Pontifícia Universidade Católica do Paraná (BR)
- Faculdades Guarulhos (BR)
- Universidad San Pedro (PE)
Publication Details
- Published
- 2025-09-22
- DOI
- https://doi.org/10.5753/wbots.2025.15181
- Primary Topic
- Open Source Software Innovations
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- Conselho Nacional de Desenvolvimento Científico e Tecnológico