Is vibe coding the future? An empirical assessment of LLM generated codes for construction safety

The emergence of vibe coding, a paradigm where non-technical users instruct Large Language Models (LLMs) to generate executable codes via natural language, presents both opportunities and risks for the construction industry. While empowering construction personnel such as the safety managers, foremen, and workers to develop tools and software, the probabilistic nature of LLMs introduces the threat of silent failures, wherein generated code compiles runs without error but executes flawed mathematical safety logic. This study empirically evaluates the reliability, software architecture, and domain-specific safety fidelity of 450 vibe-coded Python scripts generated by three frontier models, Claude 3.5 Haiku, GPT-4o-Mini, and Gemini 2.5 Flash. Utilizing a persona-driven prompt dataset (n=150) and a bifurcated evaluation pipeline comprising isolated dynamic sandboxing and an LLM-as-a-Judge, the research quantifies the limits of zero-shot vibe codes for construction safety. The findings show a statistically significant association of moderate strength between user persona and data hallucination; the generated tools invented missing safety variables more often for the less formal and less specific prompts of the field personas. Furthermore, while the models demonstrated high foundational execution viability (~85%), execution success did not indicate correct logic, and error handling was present in only 16% of scripts. Among successfully executed scripts, the study identified an ~45% overall Silent Failure Rate, with GPT-4o-Mini generating mathematically inaccurate outputs in ~56% of its functional code. The results indicate that the evaluated LLMs, used without safeguards, do not provide the deterministic reliability required for standalone safety engineering. For construction practice, they indicate that ungoverned vibe-coded tools threaten site-level safety decision-making and digital compliance workflows, and that safe adoption requires embedding LLMs within deterministic, pre-validated calculation engines under explicit organizational governance of AI-generated tools.

Authors

Institutions

Publication Details

Journal
Journal of Information Technology in Construction
Published
2026-11-17
DOI
https://doi.org/10.36680/j.itcon.2023.058
Citations
4
Primary Topic
Occupational Health and Safety Research
Type
article
Field-Weighted Citation Impact
17.42
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Is vibe coding the future? An empirical assessment of LLM generated codes for construction safety

S M Jamil Uddin
4 citations
Journal of Information Technology in Construction
Occupational Health and Safety Research
17.42
article

Is vibe coding the future? An empirical assessment of LLM generated codes for construction safety

S M Jamil Uddin
article en
4 citations

Abstract

The emergence of vibe coding, a paradigm where non-technical users instruct Large Language Models (LLMs) to generate executable codes via natural language, presents both opportunities and risks for the construction industry. While empowering construction personnel such as the safety managers, foremen, and workers to develop tools and software, the probabilistic nature of LLMs introduces the threat of silent failures, wherein generated code compiles runs without error but executes flawed mathematical safety logic. This study empirically evaluates the reliability, software architecture, and domain-specific safety fidelity of 450 vibe-coded Python scripts generated by three frontier models, Claude 3.5 Haiku, GPT-4o-Mini, and Gemini 2.5 Flash. Utilizing a persona-driven prompt dataset (n=150) and a bifurcated evaluation pipeline comprising isolated dynamic sandboxing and an LLM-as-a-Judge, the research quantifies the limits of zero-shot vibe codes for construction safety. The findings show a statistically significant association of moderate strength between user persona and data hallucination; the generated tools invented missing safety variables more often for the less formal and less specific prompts of the field personas. Furthermore, while the models demonstrated high foundational execution viability (~85%), execution success did not indicate correct logic, and error handling was present in only 16% of scripts. Among successfully executed scripts, the study identified an ~45% overall Silent Failure Rate, with GPT-4o-Mini generating mathematically inaccurate outputs in ~56% of its functional code. The results indicate that the evaluated LLMs, used without safeguards, do not provide the deterministic reliability required for standalone safety engineering. For construction practice, they indicate that ungoverned vibe-coded tools threaten site-level safety decision-making and digital compliance workflows, and that safe adoption requires embedding LLMs within deterministic, pre-validated calculation engines under explicit organizational governance of AI-generated tools.

Journal of Information Technology in ConstructionVol. 31
Colorado State University (US)
Openalex Percentile: Top 1%
Occupational Health and Safety Research
17.42
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon