Federated transfer learning for anomaly detection in HPC systems: First real-world validation on a tier-0 supercomputer

• First real-world FTL for anomaly detection on Tier-0 supercomputer. • Validated on 100 Marconi100 nodes under three learning paradigms. • FTL boosts F1 by up to 0.50 on nodes not in federated training. • Top-N vs Random-N shows diversity can outperform performance-based selection. • Enables privacy-preserving, scalable HPC fault detection without raw data sharing. High-Performance Computing (HPC) systems increasingly require intelligent, scalable anomaly detection to ensure operational reliability. However, conventional centralized approaches often struggle with data privacy constraints, poor generalization across heterogeneous nodes, and limited scalability. This study presents the first real-world application of federated transfer learning (FTL) for anomaly detection in a production-grade Tier-0 supercomputer. By combining federated learning with transfer learning, the proposed framework enables decentralized model training and personalized adaptation to unseen nodes, without accessing raw data. We validate the approach using two large-scale telemetry datasets collected from 100 nodes of the Marconi100 supercomputer, evaluating its effectiveness across supervised, semi-supervised, and unsupervised learning paradigms. Results show that FTL consistently improves anomaly detection performance on nodes that did not participate in federated training, with F1-score gains reaching up to 0.50. These improvements demonstrate the framework’s ability to generalize across non-identically distributed data and maintain detection accuracy under real-world conditions. This work establishes FTL as a scalable, privacy-preserving solution for fault detection in HPC environments. Its practical deployment on production hardware confirms its readiness for real-time monitoring applications in large-scale, heterogeneous computing systems.

Authors

Institutions

Publication Details

Journal
Expert Systems with Applications
Published
2025-09-18
DOI
https://doi.org/10.1016/j.eswa.2025.129754
Citations
2
Primary Topic
Anomaly Detection Techniques and Applications
Type
article
Field-Weighted Citation Impact
4.96
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Federated transfer learning for anomaly detection in HPC systems: First real-world validation on a tier-0 supercomputer

Andrea Borghesi, Michela Milano, Emmen Farooq
2 citations
Expert Systems with Applications
Anomaly Detection Techniques and Applications
4.96
article

Federated transfer learning for anomaly detection in HPC systems: First real-world validation on a tier-0 supercomputer

Andrea Borghesi, Michela Milano, Emmen Farooq
article en
2 citations

Abstract

• First real-world FTL for anomaly detection on Tier-0 supercomputer. • Validated on 100 Marconi100 nodes under three learning paradigms. • FTL boosts F1 by up to 0.50 on nodes not in federated training. • Top-N vs Random-N shows diversity can outperform performance-based selection. • Enables privacy-preserving, scalable HPC fault detection without raw data sharing. High-Performance Computing (HPC) systems increasingly require intelligent, scalable anomaly detection to ensure operational reliability. However, conventional centralized approaches often struggle with data privacy constraints, poor generalization across heterogeneous nodes, and limited scalability. This study presents the first real-world application of federated transfer learning (FTL) for anomaly detection in a production-grade Tier-0 supercomputer. By combining federated learning with transfer learning, the proposed framework enables decentralized model training and personalized adaptation to unseen nodes, without accessing raw data. We validate the approach using two large-scale telemetry datasets collected from 100 nodes of the Marconi100 supercomputer, evaluating its effectiveness across supervised, semi-supervised, and unsupervised learning paradigms. Results show that FTL consistently improves anomaly detection performance on nodes that did not participate in federated training, with F1-score gains reaching up to 0.50. These improvements demonstrate the framework’s ability to generalize across non-identically distributed data and maintain detection accuracy under real-world conditions. This work establishes FTL as a scalable, privacy-preserving solution for fault detection in HPC environments. Its practical deployment on production hardware confirms its readiness for real-time monitoring applications in large-scale, heterogeneous computing systems.

Expert Systems with ApplicationsVol. 298
University of Bologna (IT)
Openalex Percentile: Top 5%
Anomaly Detection Techniques and Applications
4.96
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Federated transfer learning for anomaly detection in HPC systems: First real-world validation on a tier-0 supercomputer — Andrea Borghesi, Michela Milano, et al. · Expert Systems with Applications (2025) | TGRS Research Map | TGRS