HARNESS: Holistic Resource Management for Diversely Scaled Edge Cloud Systems

Computing systems are evolving to be more ubiquitous, heterogeneous, and dynamic.Many emerging domains, such as Internet of Things (IoT), federated learning, and smart buildings, rely on a diverse edge-to-cloud continuum where the execution of applications spans various tiers of systems with significantly different computational capabilities.Computing resources in each tier, such as processing units inside of in-the-field edge devices and high-performance servers in datacenters, are handled in isolation due to scalability and resource segregation.This practice results in task mappings limited to only a subset of all available processing units, preventing an efficient overall utilization of the system.In this paper, we propose a holistic approach to capture diverse computational characteristics of edge-cloud systems with arbitrary topologies and to efficiently manage computational resources with the whole continuum in the scope.Our approach is built upon a multi-layer graph-based hardware (HW) representation and a modular performance modeling interface that can capture interactions and interference between computational resources in the system.We introduce an orchestrator mechanism that leverages the graph-based HW representation to hierarchically locate processing units to which a given set of tasks can be mapped while respecting the isolation between the computational tiers of an edge-cloud system.We demonstrate the utility of our approach on two distinct edge-cloud systems deployed in the field, improving the latency up to 47% over the best baseline with less than 2% scheduling overhead and reducing the average prediction error rate from 27.4% to 3.2%.