Input Serialization in Zero-Shot LLM Fault Diagnosis of Rooftop Units: A Leakage-Controlled Evaluation of Nine Models

Large language models are increasingly used for building fault diagnosis, but input representation is often changed alongside model adaptation or added knowledge. This study isolates serialization effects using matched rooftop unit measurements rendered as deterministic prose, fixed schema JSON, and JSON with a fixed evidence card covering all classes. Nine API models evaluated 240 windows from 39 source files in the public ORNL_RTU dataset across four diagnostic classes. Prompt development was separated from evaluation by season, and uncertainty was assessed at the source file level. Observed macro F1 differences between JSON and prose ranged from −0.082 to 0.029, and no model-specific contrast remained significant after Holm adjustment, which does not establish equivalence. Output validity nevertheless varied: JSON reduced the MiniMax invalid schema rate by 0.333, the evidence card increased Kimi diagnostic coverage by 0.508, and larger macro F1 changes coincided with changes in schema validity or abstention. Model condition macro F1 ranged from zero to 0.182, compared with 0.543 for a supervised Random Forest reference under seasonal holdout, so no evaluated zero-shot configuration is suitable for unattended fault classification. Diagnostic accuracy, schema validity, abstention, and name-level evidence grounding should therefore be evaluated jointly when selecting input representations.

Authors

Institutions

Publication Details

Journal
Intelligent infrastructure and construction
Published
2026-11-12
DOI
https://doi.org/10.3390/iic2040017
Primary Topic
Fault Detection and Control Systems
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Input Serialization in Zero-Shot LLM Fault Diagnosis of Rooftop Units: A Leakage-Controlled Evaluation of Nine Models

Hu Wei, Yiyu Cai, Kang Lai
Intelligent infrastructure and construction
Fault Detection and Control Systems
article

Input Serialization in Zero-Shot LLM Fault Diagnosis of Rooftop Units: A Leakage-Controlled Evaluation of Nine Models

Hu Wei, Yiyu Cai, Kang Lai
article en

Abstract

Large language models are increasingly used for building fault diagnosis, but input representation is often changed alongside model adaptation or added knowledge. This study isolates serialization effects using matched rooftop unit measurements rendered as deterministic prose, fixed schema JSON, and JSON with a fixed evidence card covering all classes. Nine API models evaluated 240 windows from 39 source files in the public ORNL_RTU dataset across four diagnostic classes. Prompt development was separated from evaluation by season, and uncertainty was assessed at the source file level. Observed macro F1 differences between JSON and prose ranged from −0.082 to 0.029, and no model-specific contrast remained significant after Holm adjustment, which does not establish equivalence. Output validity nevertheless varied: JSON reduced the MiniMax invalid schema rate by 0.333, the evidence card increased Kimi diagnostic coverage by 0.508, and larger macro F1 changes coincided with changes in schema validity or abstention. Model condition macro F1 ranged from zero to 0.182, compared with 0.543 for a supervised Random Forest reference under seasonal holdout, so no evaluated zero-shot configuration is suitable for unattended fault classification. Diagnostic accuracy, schema validity, abstention, and name-level evidence grounding should therefore be evaluated jointly when selecting input representations.

Intelligent infrastructure and constructionVol. 2(4)
Nanyang Technological University (SG), Nanjing University of Information Science and Technology (CN)
Fault Detection and Control Systems
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon