MG2GS: Optimizing Resource Efficiency for AI Training with Cross-MEC Job Scheduling

The increasing demand for resource-efficient AI training has positioned Mobile Edge Computing (MEC) as a pivotal component in distributed machine learning tasks. Traditional scheduling methods, however, often fail to account for the inherent characteristics of training tasks, such as periodicity and task correlation, leading to sub-optimal resource utilization. To address these limitations, we propose the Multi-Graph to Graph Scheduler (MG2GS), a novel framework designed to optimize resource allocation and scheduling in MEC environments. MG2GS employs a graph neural network-based feature extractor to capture the spatiotemporal availability of network resources, while a reinforcement learning-based scheduler ensures optimal task scheduling decisions. By incorporating task structure and long-term resource distribution, MG2GS enhances resource utilization by more than 20% compared to existing methods. Simulation results demonstrate its effectiveness in increasing the number of scheduled tasks and improving overall resource efficiency for distributed AI model training.

Authors

Institutions

Publication Details

Published
2024-12-13
DOI
https://doi.org/10.1109/hpcc64274.2024.00215
Primary Topic
Scheduling and Optimization Algorithms
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

MG2GS: Optimizing Resource Efficiency for AI Training with Cross-MEC Job Scheduling

Xiangyang Gong, Wendong Wang, Ye Tian, Jiahui Hu et al.
Scheduling and Optimization Algorithms
article

MG2GS: Optimizing Resource Efficiency for AI Training with Cross-MEC Job Scheduling

Xiangyang Gong, Wendong Wang, Ye Tian, Jiahui Hu, Yannan Hu, Zeming Gao
article en

Abstract

The increasing demand for resource-efficient AI training has positioned Mobile Edge Computing (MEC) as a pivotal component in distributed machine learning tasks. Traditional scheduling methods, however, often fail to account for the inherent characteristics of training tasks, such as periodicity and task correlation, leading to sub-optimal resource utilization. To address these limitations, we propose the Multi-Graph to Graph Scheduler (MG2GS), a novel framework designed to optimize resource allocation and scheduling in MEC environments. MG2GS employs a graph neural network-based feature extractor to capture the spatiotemporal availability of network resources, while a reinforcement learning-based scheduler ensures optimal task scheduling decisions. By incorporating task structure and long-term resource distribution, MG2GS enhances resource utilization by more than 20% compared to existing methods. Simulation results demonstrate its effectiveness in increasing the number of scheduled tasks and improving overall resource efficiency for distributed AI model training.

State Key Laboratory of Networking and Switching Technology
Decent work and economic growth
Openalex Percentile: Top 55%
Scheduling and Optimization Algorithms
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.