A Catalog of Data Smells for Coding Tasks

Large Language Models (LLMs) are increasingly becoming fundamental in supporting software developers in coding tasks. The massive datasets used for training LLMs are often collected automatically, leading to the introduction of data smells. Previous work addressed this issue by using quality filters to handle some specific smells. Still, the literature lacks a systematic catalog of the data smells for coding tasks currently known. This article presents a Systematic Literature Review (SLR) focused on articles that introduce LLMs for coding tasks. We first extracted the quality filters adopted for training and testing such LLMs, inferred the root problem behind their adoption (data smells for coding tasks), and defined a taxonomy of such smells. Our results highlight discrepancies in the adoption of quality filters between pre-training and fine-tuning stages and across different coding tasks, shedding light on areas for improvement in LLM-based software development support.

Authors

Institutions

Publication Details

Journal
ACM Transactions on Software Engineering and Methodology
Published
2024-12-10
DOI
https://doi.org/10.1145/3707457
Citations
4
Primary Topic
Software Engineering Research
Type
article
Field-Weighted Citation Impact
3.12

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

A Catalog of Data Smells for Coding Tasks

Rocco Oliveto, Simone Scalabrino, Antonio Vitale
4 citations
ACM Transactions on Software Engineering and Methodology
Software Engineering Research
3.12
article

A Catalog of Data Smells for Coding Tasks

Rocco Oliveto, Simone Scalabrino, Antonio Vitale
article en
4 citations

Abstract

Large Language Models (LLMs) are increasingly becoming fundamental in supporting software developers in coding tasks. The massive datasets used for training LLMs are often collected automatically, leading to the introduction of data smells. Previous work addressed this issue by using quality filters to handle some specific smells. Still, the literature lacks a systematic catalog of the data smells for coding tasks currently known. This article presents a Systematic Literature Review (SLR) focused on articles that introduce LLMs for coding tasks. We first extracted the quality filters adopted for training and testing such LLMs, inferred the root problem behind their adoption (data smells for coding tasks), and defined a taxonomy of such smells. Our results highlight discrepancies in the adoption of quality filters between pre-training and fine-tuning stages and across different coding tasks, shedding light on areas for improvement in LLM-based software development support.

ACM Transactions on Software Engineering and MethodologyVol. 34(4)
University of Molise (IT), Politecnico di Torino (IT)
European Commission
Industry, innovation and infrastructure
Openalex Percentile: Top 7%
Software Engineering Research
3.12
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

A Catalog of Data Smells for Coding Tasks — Rocco Oliveto, Simone Scalabrino, et al. · ACM Transactions on Software Engineering and Methodology (2024) | TGRS Research Map | TGRS