Uncertainty-Aware Large Language Model Framework for Failure Mode and Effects Analysis Severity Prediction: A Case Study on Autonomous Truck Systems

Autonomous heavy-duty trucks require high safety assurance across complex operational domains, necessitating comprehensive risk assessment of their intricate multi-subsystem interdependencies. Failure mode and effects analysis (FMEA) is a systematic risk assessment methodology widely employed across safety-critical industries, yet it relies heavily on manual, experience-dependent expert judgment and is difficult to apply consistently at scale. Large language models (LLMs) offer an opportunity to support this language-centered workflow. This study presents a framework that couples a fine-tuned LLM with an uncertainty-aware post-filter (UAPF) as a severity-scoring assistant and consistency checker within the human-led FMEA process. We employed quantized low-rank adaptation to adapt the Llama-3.1-8B-Instruct LLM to a comprehensive dataset of over 38,000 FMEA records of autonomous truck systems. UAPF refines the predicted severity according to the model’s predictive uncertainty, and rows whose uncertainty exceeds a validation-set threshold are escalated to expert review. On the held-out test set, the fine-tuned LLM reaches 97.75% accuracy, and UAPF raises it to 97.83% while improving calibration. With a 5% expert-review budget, the escalation rule intercepts 78.2% of the remaining model errors. Under a stricter evaluation in which entire subsystems are held out, accuracy is 94.15%, characterizing transfer to unseen functional domains. To the best of the authors’ knowledge, this work is among the first to combine a fine-tuned LLM with uncertainty-aware post-processing for FMEA severity assessment of autonomous trucks.

Authors

Institutions

Publication Details

Journal
Transportation Research Record Journal of the Transportation Research Board
Published
2026-11-20
DOI
https://doi.org/10.1177/03611981261502431
Primary Topic
Safety Systems Engineering in Autonomy
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Uncertainty-Aware Large Language Model Framework for Failure Mode and Effects Analysis Severity Prediction: A Case Study on Autonomous Truck Systems

Zejiang Wang, Tianyang Cui
5 links
Transportation Research Record Journal of the Transportation Research Board
Safety Systems Engineering in Autonomy
article

Uncertainty-Aware Large Language Model Framework for Failure Mode and Effects Analysis Severity Prediction: A Case Study on Autonomous Truck Systems

Zejiang Wang, Tianyang Cui
article en

Abstract

Autonomous heavy-duty trucks require high safety assurance across complex operational domains, necessitating comprehensive risk assessment of their intricate multi-subsystem interdependencies. Failure mode and effects analysis (FMEA) is a systematic risk assessment methodology widely employed across safety-critical industries, yet it relies heavily on manual, experience-dependent expert judgment and is difficult to apply consistently at scale. Large language models (LLMs) offer an opportunity to support this language-centered workflow. This study presents a framework that couples a fine-tuned LLM with an uncertainty-aware post-filter (UAPF) as a severity-scoring assistant and consistency checker within the human-led FMEA process. We employed quantized low-rank adaptation to adapt the Llama-3.1-8B-Instruct LLM to a comprehensive dataset of over 38,000 FMEA records of autonomous truck systems. UAPF refines the predicted severity according to the model’s predictive uncertainty, and rows whose uncertainty exceeds a validation-set threshold are escalated to expert review. On the held-out test set, the fine-tuned LLM reaches 97.75% accuracy, and UAPF raises it to 97.83% while improving calibration. With a 5% expert-review budget, the escalation rule intercepts 78.2% of the remaining model errors. Under a stricter evaluation in which entire subsystems are held out, accuracy is 94.15%, characterizing transfer to unseen functional domains. To the best of the authors’ knowledge, this work is among the first to combine a fine-tuned LLM with uncertainty-aware post-processing for FMEA severity assessment of autonomous trucks.

Transportation Research Record Journal of the Transportation Research Board
The University of Texas at Dallas (US)
Industry, innovation and infrastructure
Safety Systems Engineering in Autonomy
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon