Investigations on Disk Performance Indices and Their Statistical Learning for Storage Cluster Disk Fail-Slow Detection

Disks are critical components of data center storage systems, and their health directly affects system stability. However, when a disk enters a fail-slow state, it continues to function but experiences significant performance degradation, leading to slower response time and negatively affecting business service quality. Being different from disk failure, the fail-slow state is hard to be detected by using traditional disk failure detection methods. The scenario of fail-slow is very complex, therefore a flexible fail-slow processing framework is required, and the detection must be carried out on the premise of ensuring the reliability and high availability of the storage system. To address this challenge, we propose a non-invasive and adaptive method for fail-slow disk detection in storage clusters in this paper. Firstly, a latency prediction model is proposed to intelligently correlate disk performance metrics from a database. Based on this, we propose a health index-based monitoring strategy for fail-slow disk states and cluster health assessment. After deployment in a real production environment, this method has monitored thousands of disks, effectively identifying dozens of fail-slow disks and mitigating business risks caused by hardware issues.