Real-time AI-powered monitoring for energy-efficient scheduling in multi-node heterogeneous systems

Load balancing is critical for maintaining computing systems’ stability and achieving optimal performance. Its significance is widely recognized across different computing fields, particularly in the context of heterogeneous systems. These systems comprise computing devices with varying computational capabilities and architectures, each optimized for specific workloads. This heterogeneity introduces dynamic resource constraints, architectural mismatches, and unpredictable task-device affinity, which aggravates the challenges of load balancing. This paper presents an AI-driven load balancing solution for real-time distributed heterogeneous systems. Our approach continuously monitors the system state, capturing key factors that influence performance, such as task and device characteristics. Leveraging AI-based models, it computes a dynamic load index for each device based on the collected data. Using these load estimations, the method predicts potential imbalances through a novel imbalance metric and proactively schedules incoming applications to the most suitable devices, ensuring system-wide balance. To validate our approach, we first evaluated the prediction models by comparing a variety of machine learning algorithms with device-specific deep learning models, with the latter achieving superior accuracy. We then compared our method against widely used scheduling techniques across diverse workloads. The results show that our approach achieves more balanced workload distribution, faster execution, higher throughput, improved resource utilization, and reduced energy consumption across all scenarios, showcasing its adaptability to dynamic conditions and its applicability in real-world settings.