How Readable Is LLM-Generated Code Snippets? A Comparison of ChatGPT, DeepSeek, and Gemini

Developers often search for reusable code snippets on the Web. With the increasing adoption of Large Language Models (LLMs) to support programming tasks and the growing number of available models, this work proposes an evaluation of the readability of 981 code snippets generated by ChatGPT, DeepSeek, and Gemini (327 by each model) by analyzing the warnings detected by static analysis tools such as SonarLint. Additionally, we present a preliminary approach that combines SonarLint recommendations with LLMs to automatically refactor code snippets with the goal of improving code readability. The results show that ChatGPT produces code with fewer readability warnings according to SonarLint. All three LLMs were also able to remove readability warnings in more than 60% of the affected code snippets. However, challenges remain when combining LLMs with static analysis tools, particularly in understanding the context of certain rules and avoiding the removal of relevant code. The insights from this study reveal opportunities for deeper integration between static analysis tools and LLMs.

Authors

Publication Details

Published
2025-09-18
DOI
https://doi.org/10.5753/vem.2025.14275
Primary Topic
Topic Modeling
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

How Readable Is LLM-Generated Code Snippets? A Comparison of ChatGPT, DeepSeek, and Gemini

Marcelo de Almeida Maia, Carlos Eduardo C. Dantas, Giovanna Fernandes
Topic Modeling
article

How Readable Is LLM-Generated Code Snippets? A Comparison of ChatGPT, DeepSeek, and Gemini

Marcelo de Almeida Maia, Carlos Eduardo C. Dantas, Giovanna Fernandes
article en

Abstract

Developers often search for reusable code snippets on the Web. With the increasing adoption of Large Language Models (LLMs) to support programming tasks and the growing number of available models, this work proposes an evaluation of the readability of 981 code snippets generated by ChatGPT, DeepSeek, and Gemini (327 by each model) by analyzing the warnings detected by static analysis tools such as SonarLint. Additionally, we present a preliminary approach that combines SonarLint recommendations with LLMs to automatically refactor code snippets with the goal of improving code readability. The results show that ChatGPT produces code with fewer readability warnings according to SonarLint. All three LLMs were also able to remove readability warnings in more than 60% of the affected code snippets. However, challenges remain when combining LLMs with static analysis tools, particularly in understanding the context of certain rules and avoiding the removal of relevant code. The insights from this study reveal opportunities for deeper integration between static analysis tools and LLMs.

Openalex Percentile: Top 87%
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.