Auto-tuning for HPC storage stack: an optimization perspective

Abstract Storage stack layers in high-performance computing (HPC) systems offer many tunable parameters controlling I/O behaviors and underlying file system settings. The setting of these parameters plays a decisive role in I/O performance. Nevertheless, the increasing complexity of data operations and storage architectures makes identifying a set of well-performing configurations a challenge. Auto-tuning is a promising technology. This paper presents a comprehensive survey on "Auto-tuning in HPC I/O". We expound a general storage structure based on a general storage stack and critical elements of auto-tuning, and categorize related studies according to the way of tuning. On the basis of the order in which the approaches were applied, we introduce the specific works of each approach in detail, and summarize and compare the pros and cons of these approaches. Through a comprehensive and in-depth study of existing research, we elaborate on the development history of auto-tuning technology in HPC I/O, analyze the current situation, and provide guidance for optimization technology in the future.