Hidden Faults Under Partial Sensor Acquisition: Revisiting the AI4I-PMDI Benchmark and the Value of Machine History

This study audits AI4I-PMDI, a public predictive maintenance benchmark with inspection-dependent missing values built on the AI4I 2020 data, and finds that it is easier than the same data with nothing missing. Its design assigns every failure record to an inspection type that measures that failure's causal sensors, so the published file contains no hidden faults, that is, faults recorded under an inspection that did not measure all the sensors their cause depends on. XGBoost reaches a PR-AUC (the area under the precision-recall curve, which summarises how well faults are ranked above normal records) of 0.878 on the published schedule but only 0.778 when no values are missing. The study separates random dropout, the usual way of simulating missing sensors, from schedule-dependent missingness, in which the inspection plan can leave out exactly the sensors that respond to a fault. To test the second case, the benchmark was rebuilt from the AI4I 2020 values so that a failure can fall under any inspection type. All models were evaluated with five-fold cross-validation grouped by machine, at fixed false-alarm rates (the share of normal records that trigger an alarm), and compared with a paired bootstrap over machines. Under the realistic schedule PR-AUC falls to 0.452, and at one false alarm per 100 normal records only 20% of hidden faults are detected; a model developed on the published schedule detects just 4%. Imputation, missingness indicators and separate models for each inspection type do not close this gap. Adding the last earlier reading of each sensor from the same machine raises PR-AUC by 0.130 (95% CI 0.088 to 0.165) and hidden-fault recall from 20% to 45%. The gain holds across five classifiers, and history features with XGBoost outperform GRU-D, a recurrent model designed for missing observations, on this benchmark. On C-MAPSS FD001 the gain is in the same direction; on MetroPT-3 history ranks windows better but detects leaks later; a combined rule recovers the leaks that history alone misses but does not outperform current readings alone. The benefit comes from the machine's history rather than from the exact way it is encoded. Benchmarks with schedule-dependent missing values should therefore also be evaluated under schedules that can hide faults, and monitoring systems should keep each sensor's last reading for every machine.

Authors

Institutions

Publication Details

Journal
Journal of Scientific Reports
Published
2026-11-21
DOI
https://doi.org/10.58970/jsr.1258
Primary Topic
Machine Fault Diagnosis Techniques
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Hidden Faults Under Partial Sensor Acquisition: Revisiting the AI4I-PMDI Benchmark and the Value of Machine History

Chernor Illah Jalloh, Christiana Lebbie, Marvin Sinclair Mkaliainga, José da Conceição Ernesto Langa Júnior
Journal of Scientific Reports
Machine Fault Diagnosis Techniques
article

Hidden Faults Under Partial Sensor Acquisition: Revisiting the AI4I-PMDI Benchmark and the Value of Machine History

Chernor Illah Jalloh, Christiana Lebbie, Marvin Sinclair Mkaliainga, José da Conceição Ernesto Langa Júnior
article en

Abstract

This study audits AI4I-PMDI, a public predictive maintenance benchmark with inspection-dependent missing values built on the AI4I 2020 data, and finds that it is easier than the same data with nothing missing. Its design assigns every failure record to an inspection type that measures that failure's causal sensors, so the published file contains no hidden faults, that is, faults recorded under an inspection that did not measure all the sensors their cause depends on. XGBoost reaches a PR-AUC (the area under the precision-recall curve, which summarises how well faults are ranked above normal records) of 0.878 on the published schedule but only 0.778 when no values are missing. The study separates random dropout, the usual way of simulating missing sensors, from schedule-dependent missingness, in which the inspection plan can leave out exactly the sensors that respond to a fault. To test the second case, the benchmark was rebuilt from the AI4I 2020 values so that a failure can fall under any inspection type. All models were evaluated with five-fold cross-validation grouped by machine, at fixed false-alarm rates (the share of normal records that trigger an alarm), and compared with a paired bootstrap over machines. Under the realistic schedule PR-AUC falls to 0.452, and at one false alarm per 100 normal records only 20% of hidden faults are detected; a model developed on the published schedule detects just 4%. Imputation, missingness indicators and separate models for each inspection type do not close this gap. Adding the last earlier reading of each sensor from the same machine raises PR-AUC by 0.130 (95% CI 0.088 to 0.165) and hidden-fault recall from 20% to 45%. The gain holds across five classifiers, and history features with XGBoost outperform GRU-D, a recurrent model designed for missing observations, on this benchmark. On C-MAPSS FD001 the gain is in the same direction; on MetroPT-3 history ranks windows better but detects leaks later; a combined rule recovers the leaks that history alone misses but does not outperform current readings alone. The benefit comes from the machine's history rather than from the exact way it is encoded. Benchmarks with schedule-dependent missing values should therefore also be evaluated under schedules that can hide faults, and monitoring systems should keep each sensor's last reading for every machine.

Journal of Scientific ReportsVol. 15(1)
Huazhong University of Science and Technology (CN)
Machine Fault Diagnosis Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon