MG2GS: Optimizing Resource Efficiency for AI Training with Cross-MEC Job Scheduling
The increasing demand for resource-efficient AI training has positioned Mobile Edge Computing (MEC) as a pivotal component in distributed machine learning tasks. Traditional scheduling methods, however, often fail to account for the inherent characteristics of training tasks, such as periodicity and task correlation, leading to sub-optimal resource utilization. To address these limitations, we propose the Multi-Graph to Graph Scheduler (MG2GS), a novel framework designed to optimize resource allocation and scheduling in MEC environments. MG2GS employs a graph neural network-based feature extractor to capture the spatiotemporal availability of network resources, while a reinforcement learning-based scheduler ensures optimal task scheduling decisions. By incorporating task structure and long-term resource distribution, MG2GS enhances resource utilization by more than 20% compared to existing methods. Simulation results demonstrate its effectiveness in increasing the number of scheduled tasks and improving overall resource efficiency for distributed AI model training.
Authors
- Xiangyang Gong (ORCID: https://orcid.org/0000-0002-0631-9747)
- Wendong Wang (ORCID: https://orcid.org/0000-0002-6418-8087)
- Ye Tian (ORCID: https://orcid.org/0000-0001-6700-4098)
- Jiahui Hu (ORCID: https://orcid.org/0000-0001-8771-7474)
- Yannan Hu
- Zeming Gao
Institutions
- State Key Laboratory of Networking and Switching Technology
Publication Details
- Published
- 2024-12-13
- DOI
- https://doi.org/10.1109/hpcc64274.2024.00215
- Primary Topic
- Scheduling and Optimization Algorithms
- Type
- article
- Field-Weighted Citation Impact
- 0.00