Naming the Hazard Category, Missing the Accident Mechanism: Fatal-Accident Coverage of LLM-Drafted Construction Risk Assessments Generated from Work Names Alone

Large language models (LLMs) can draft construction risk assessments from work names, but their coverage of fatal-accident mechanisms is uncertain. We evaluated 2097 hazards generated by GPT-5.6 Sol and Claude Fable 5 from work-category, work, and unit-work names alone. The reference was 1687 hazard factors for 113 unit works from the Korea Occupational Safety and Health Agency’s fatality-derived table (2016–2021). Two LLM judges scored hazard coverage and control adequacy. Two external experts independently rated 198 sampled items (judge–expert quadratic-weighted κ = 0.65–0.84). At the boundary between no and partial coverage, unweighted κ was only 0.37–0.71. Across the four service–judge combinations, 66.9–77.2% of the reference hazard factors were at least partially covered, but only 25.5–29.0% were fully covered. Full coverage is the primary and more reliable result, and it changed little under alternative matching rules. Shared omissions involved equipment states, work stages, and worker positions. Key preventive measures accompanied 95–97% of fully covered reference–draft pairs but 52–75% of partially covered pairs, depending on the judge; part of this association is built into the rubric. Of 5542 legal citations, 4689 cited the Rules on Occupational Safety and Health Standards; 99.6% of these named an existing article, and formal number–title agreement was 94.6% (GPT-5.6 Sol) and 95.9% (Claude Fable 5). All estimates describe one generation per service under the tested conditions and should not be generalized to the models’ overall performance. Such drafts require review focused on specific accident conditions and controls.

Authors

Institutions

Publication Details

Journal
Buildings
Published
2026-11-16
DOI
https://doi.org/10.3390/buildings16224554
Primary Topic
Occupational Health and Safety Research
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Naming the Hazard Category, Missing the Accident Mechanism: Fatal-Accident Coverage of LLM-Drafted Construction Risk Assessments Generated from Work Names Alone

Yongha Lee, Jaewook Lee, Jong-Ho Lee
5 links
Buildings
Occupational Health and Safety Research
article

Naming the Hazard Category, Missing the Accident Mechanism: Fatal-Accident Coverage of LLM-Drafted Construction Risk Assessments Generated from Work Names Alone

Yongha Lee, Jaewook Lee, Jong-Ho Lee
article en

Abstract

Large language models (LLMs) can draft construction risk assessments from work names, but their coverage of fatal-accident mechanisms is uncertain. We evaluated 2097 hazards generated by GPT-5.6 Sol and Claude Fable 5 from work-category, work, and unit-work names alone. The reference was 1687 hazard factors for 113 unit works from the Korea Occupational Safety and Health Agency’s fatality-derived table (2016–2021). Two LLM judges scored hazard coverage and control adequacy. Two external experts independently rated 198 sampled items (judge–expert quadratic-weighted κ = 0.65–0.84). At the boundary between no and partial coverage, unweighted κ was only 0.37–0.71. Across the four service–judge combinations, 66.9–77.2% of the reference hazard factors were at least partially covered, but only 25.5–29.0% were fully covered. Full coverage is the primary and more reliable result, and it changed little under alternative matching rules. Shared omissions involved equipment states, work stages, and worker positions. Key preventive measures accompanied 95–97% of fully covered reference–draft pairs but 52–75% of partially covered pairs, depending on the judge; part of this association is built into the rubric. Of 5542 legal citations, 4689 cited the Rules on Occupational Safety and Health Standards; 99.6% of these named an existing article, and formal number–title agreement was 94.6% (GPT-5.6 Sol) and 95.9% (Claude Fable 5). All estimates describe one generation per service under the tested conditions and should not be generalized to the models’ overall performance. Such drafts require review focused on specific accident conditions and controls.

BuildingsVol. 16(22)
Gachon University (KR), Korea Institute of Civil Engineering and Building Technology (KR)
Ministry of Land, Infrastructure and Transport
Occupational Health and Safety Research
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon