Machine Learning Versus Threshold Monitoring for Disk Failure Prediction in Data Centers: A Time-Aware Comparison on Public Back blaze Data

Predicting hard-disk failure from SMART attributes is widely studied, but reported results are often hard to compare because evaluation order, sampling of healthy drives and the choice of baseline differ. This paper compares machine-learning failure prediction with threshold monitoring on the public Backblaze Drive Stats data for one drive model (Seagate ST4000DM000, 2016–2017; 1997 failed and 1598 sampled healthy drives). The task is to predict failure within the next 7 days. Training, validation and test periods are strictly chronological with purged boundaries, healthy drives are reweighted to the full fleet, and a static rule and a rule tuned on validation data are compared with Logistic Regression, Random Forest and gradient boosting using fleet-weighted metrics, drive-level warning measures and stratified bootstrap intervals. The tuned rule has the highest PR-AUC (0.089, against 0.078 for Logistic Regression and 0.050 for Random Forest). At the validation-selected operating point the learned models give higher F1 (0.114 to 0.136 against 0.025 and 0.047) and a false-positive rate that is several times lower; Logistic Regression warns of 47 % of 510 failing drives in the test period, similar to the tuned rule (48 %), with about a quarter of the alert days. About 42.9 % of failing drives show no sign in the main SMART error attributes in their last 30 days. Machine learning thus mainly reduces false alarms in this data, and a tuned threshold rule is a strong baseline. The results are limited to one drive model and one fleet.

Authors

Institutions

Publication Details

Journal
Indian Journal of Computer Science and Technology
Published
2026-11-13
DOI
https://doi.org/10.59256/indjcst.20260503096
Primary Topic
Advanced Data Storage Technologies
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Machine Learning Versus Threshold Monitoring for Disk Failure Prediction in Data Centers: A Time-Aware Comparison on Public Back blaze Data

Rajesh Debnath, Yagya Dutt, Chauhan Shubham, Devi Raksha et al.
Indian Journal of Computer Science and Technology
Advanced Data Storage Technologies
article

Machine Learning Versus Threshold Monitoring for Disk Failure Prediction in Data Centers: A Time-Aware Comparison on Public Back blaze Data

Rajesh Debnath, Yagya Dutt, Chauhan Shubham, Devi Raksha, Kumar Ajay
article en

Abstract

Predicting hard-disk failure from SMART attributes is widely studied, but reported results are often hard to compare because evaluation order, sampling of healthy drives and the choice of baseline differ. This paper compares machine-learning failure prediction with threshold monitoring on the public Backblaze Drive Stats data for one drive model (Seagate ST4000DM000, 2016–2017; 1997 failed and 1598 sampled healthy drives). The task is to predict failure within the next 7 days. Training, validation and test periods are strictly chronological with purged boundaries, healthy drives are reweighted to the full fleet, and a static rule and a rule tuned on validation data are compared with Logistic Regression, Random Forest and gradient boosting using fleet-weighted metrics, drive-level warning measures and stratified bootstrap intervals. The tuned rule has the highest PR-AUC (0.089, against 0.078 for Logistic Regression and 0.050 for Random Forest). At the validation-selected operating point the learned models give higher F1 (0.114 to 0.136 against 0.025 and 0.047) and a false-positive rate that is several times lower; Logistic Regression warns of 47 % of 510 failing drives in the test period, similar to the tuned rule (48 %), with about a quarter of the alert days. About 42.9 % of failing drives show no sign in the main SMART error attributes in their last 30 days. Machine learning thus mainly reduces false alarms in this data, and a tuned threshold rule is a strong baseline. The results are limited to one drive model and one fleet.

Indian Journal of Computer Science and Technology
Himachal Pradesh University (IN)
Advanced Data Storage Technologies
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon