Uncertainty-Aware Large Language Model Framework for Failure Mode and Effects Analysis Severity Prediction: A Case Study on Autonomous Truck Systems
Autonomous heavy-duty trucks require high safety assurance across complex operational domains, necessitating comprehensive risk assessment of their intricate multi-subsystem interdependencies. Failure mode and effects analysis (FMEA) is a systematic risk assessment methodology widely employed across safety-critical industries, yet it relies heavily on manual, experience-dependent expert judgment and is difficult to apply consistently at scale. Large language models (LLMs) offer an opportunity to support this language-centered workflow. This study presents a framework that couples a fine-tuned LLM with an uncertainty-aware post-filter (UAPF) as a severity-scoring assistant and consistency checker within the human-led FMEA process. We employed quantized low-rank adaptation to adapt the Llama-3.1-8B-Instruct LLM to a comprehensive dataset of over 38,000 FMEA records of autonomous truck systems. UAPF refines the predicted severity according to the model’s predictive uncertainty, and rows whose uncertainty exceeds a validation-set threshold are escalated to expert review. On the held-out test set, the fine-tuned LLM reaches 97.75% accuracy, and UAPF raises it to 97.83% while improving calibration. With a 5% expert-review budget, the escalation rule intercepts 78.2% of the remaining model errors. Under a stricter evaluation in which entire subsystems are held out, accuracy is 94.15%, characterizing transfer to unseen functional domains. To the best of the authors’ knowledge, this work is among the first to combine a fine-tuned LLM with uncertainty-aware post-processing for FMEA severity assessment of autonomous trucks.
Authors
- Zejiang Wang (ORCID: https://orcid.org/0000-0001-9422-4846)
- Tianyang Cui (ORCID: https://orcid.org/0000-0002-3155-9697)
Institutions
- The University of Texas at Dallas (US)
Publication Details
- Journal
- Transportation Research Record Journal of the Transportation Research Board
- Published
- 2026-11-20
- DOI
- https://doi.org/10.1177/03611981261502431
- Primary Topic
- Safety Systems Engineering in Autonomy
- Type
- article
- Field-Weighted Citation Impact
- 0.00