Characterizing and Classifying Self-Admitted Technical Debt in Infrastructure-as-Code

Infrastructure-as-Code (IaC) has become a foundational practice in modern cloud-based software systems, enabling developers to declaratively specify, provision, and manage infrastructure through code. By treating infrastructure as a first-class, version-controlled artifact, IaC represents a paradigm shift in how infrastructure is designed, evolved, and maintained. This shift also introduces new forms of complexity, as infrastructure definitions rapidly evolve alongside cloud platforms and deployment pipelines. As a result, IaC developers typically rely on temporary workarounds and deferred decisions, a phenomenon commonly referred to as technical debt (TD). To document and manage such debt, IaC developers introduce Self-Admitted Technical Debt (SATD) through explicit code comments (e.g., TODO, FIXME) that mark known issues or deferred improvements. Although prior research has extensively examined SATD in traditional software artifacts, little is known about its characteristics in IaC. Due to the structural coupling and expressive nature of IaC code, a single SATD comment can reflect multiple concerns. In this paper, we characterize the types and patterns of SATD in IaC artifacts and develop an automated multi-label classification approach. We conduct a qualitative analysis of 680 SATD instances from 76 Terraform-based projects, as Terraform is a widely used IaC tool. This analysis aims to ( i ) identify SATD types and underlying rationales, ( ii ) automate their classification, and ( iii ) analyze where and under which instance characteristics LLMs succeed or fail using Instance Space Analysis (ISA). Our findings reveal that IaC-specific SATD exhibits a structured taxonomy consisting of eight high-level categories and 25 sub-categories. Among these, IaC code debt is the most prevalent category, accounting for 86% of all instances, and covers improper patterns, incomplete attribute specifications, and deferred features. In addition, 81% of the analyzed SATD instances in IaC span at least two categories, highlighting the substantial degree of overlap among SATD types within IaC artifacts. Building on this, we evaluate Large Language Models (LLMs) for SATD classification in multi-label settings, where Gemini achieves the highest weighted F1-score under Zero-shot prompting (0.79), while DeepSeek achieves the highest under Few-shot prompting (0.83), outperforming traditional machine learning baselines. Using ISA, we show that classification performance varies systematically across SATD instances with different structural dependencies and comment complexities. These findings motivate rethinking SATD in IaC as a multi-dimensional phenomenon that intertwines configuration, security, and deployment concerns, requiring specialized tool support and processes.

Authors

Institutions

Publication Details

Journal
ACM Transactions on Software Engineering and Methodology
Published
2026-09-17
DOI
https://doi.org/10.1145/3848031
Primary Topic
Software System Performance and Reliability
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
article

Characterizing and Classifying Self-Admitted Technical Debt in Infrastructure-as-Code

Eman Abdullah AlOmar, Ali Ouni, Mahi Begoug, Ala Ajjel
ACM Transactions on Software Engineering and Methodology
Software System Performance and Reliability
article

Characterizing and Classifying Self-Admitted Technical Debt in Infrastructure-as-Code

Eman Abdullah AlOmar, Ali Ouni, Mahi Begoug, Ala Ajjel
article en

Abstract

Infrastructure-as-Code (IaC) has become a foundational practice in modern cloud-based software systems, enabling developers to declaratively specify, provision, and manage infrastructure through code. By treating infrastructure as a first-class, version-controlled artifact, IaC represents a paradigm shift in how infrastructure is designed, evolved, and maintained. This shift also introduces new forms of complexity, as infrastructure definitions rapidly evolve alongside cloud platforms and deployment pipelines. As a result, IaC developers typically rely on temporary workarounds and deferred decisions, a phenomenon commonly referred to as technical debt (TD). To document and manage such debt, IaC developers introduce Self-Admitted Technical Debt (SATD) through explicit code comments (e.g., TODO, FIXME) that mark known issues or deferred improvements. Although prior research has extensively examined SATD in traditional software artifacts, little is known about its characteristics in IaC. Due to the structural coupling and expressive nature of IaC code, a single SATD comment can reflect multiple concerns. In this paper, we characterize the types and patterns of SATD in IaC artifacts and develop an automated multi-label classification approach. We conduct a qualitative analysis of 680 SATD instances from 76 Terraform-based projects, as Terraform is a widely used IaC tool. This analysis aims to ( i ) identify SATD types and underlying rationales, ( ii ) automate their classification, and ( iii ) analyze where and under which instance characteristics LLMs succeed or fail using Instance Space Analysis (ISA). Our findings reveal that IaC-specific SATD exhibits a structured taxonomy consisting of eight high-level categories and 25 sub-categories. Among these, IaC code debt is the most prevalent category, accounting for 86% of all instances, and covers improper patterns, incomplete attribute specifications, and deferred features. In addition, 81% of the analyzed SATD instances in IaC span at least two categories, highlighting the substantial degree of overlap among SATD types within IaC artifacts. Building on this, we evaluate Large Language Models (LLMs) for SATD classification in multi-label settings, where Gemini achieves the highest weighted F1-score under Zero-shot prompting (0.79), while DeepSeek achieves the highest under Few-shot prompting (0.83), outperforming traditional machine learning baselines. Using ISA, we show that classification performance varies systematically across SATD instances with different structural dependencies and comment complexities. These findings motivate rethinking SATD in IaC as a multi-dimensional phenomenon that intertwines configuration, security, and deployment concerns, requiring specialized tool support and processes.

ACM Transactions on Software Engineering and Methodology
Stevens Institute of Technology (US), Concordia University (CA), École de Technologie Supérieure (CA)
Industry, innovation and infrastructure
Openalex Percentile: Top 8%
Software System Performance and Reliability
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.