Towards a High Level Linter for Data Science

Due to its interdisciplinary nature, the development of data science code is subject to a wide range of potential mistakes that can easily compromise the final results. Several tools have been proposed that can help the data scientist in identifying the most common, low level programming issues. We discuss the steps needed to implement a tool that is rather meant to focus on higher level errors that are specific of the data science pipeline. To this end, we propose a static analysis assigning ad hoc abstract datatypes to the program variables, which are then checked for consistency when calling functions defined in data science libraries. By adopting a descriptive (rather than prescriptive) abstract type system, we obtain a linter tool reporting data science related code smells. While being still work in progress, the current prototype is able to identify and report the code smells contained in several examples of questionable data science code.

Authors

Institutions

Publication Details

Published
2024-10-17
DOI
https://doi.org/10.1145/3689609.3689996
Citations
2
Primary Topic
Advanced Malware Detection Techniques
Type
article
Field-Weighted Citation Impact
0.70
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Towards a High Level Linter for Data Science

Greta Dolcetti, Agostino Cortesi, Enea Zaffanella, Caterina Urban
2 citations
Advanced Malware Detection Techniques
0.70
article

Towards a High Level Linter for Data Science

Greta Dolcetti, Agostino Cortesi, Enea Zaffanella, Caterina Urban
article en
2 citations

Abstract

Due to its interdisciplinary nature, the development of data science code is subject to a wide range of potential mistakes that can easily compromise the final results. Several tools have been proposed that can help the data scientist in identifying the most common, low level programming issues. We discuss the steps needed to implement a tool that is rather meant to focus on higher level errors that are specific of the data science pipeline. To this end, we propose a static analysis assigning ad hoc abstract datatypes to the program variables, which are then checked for consistency when calling functions defined in data science libraries. By adopting a descriptive (rather than prescriptive) abstract type system, we obtain a linter tool reporting data science related code smells. While being still work in progress, the current prototype is able to identify and report the code smells contained in several examples of questionable data science code.

University of Parma (IT), Ca' Foscari University of Venice (IT)
Openalex Percentile: Top 30%
Advanced Malware Detection Techniques
0.70
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Towards a High Level Linter for Data Science — Greta Dolcetti, Agostino Cortesi, et al. · (2024) | TGRS Research Map | TGRS