Poster: Simulation-Guided Strategy Generation for Intent-Aware Distributed LLMs Training

Training distributed large language models (LLMs) for diverse user intents is challenging due to a high-dimensional, hybrid strategy space and heterogeneous workloads. We present SG<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup>, a simulation-guided strategy generation framework that tackles intent-aware distributed LLM training by unifying Bayesian Optimization with a simulation-based evaluation loop. By leveraging near-realistic performance and intent-satisfaction metrics in simulation, SG<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> iteratively refines candidate training strategies before real deployment, significantly reducing exploration cost and failure risk. Experiments on heterogeneous LLM training workloads show that SG<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> consistently achieves higher intent-satisfaction rates and better multi-objective trade-offs than baselines, demonstrating its effectiveness for intent-aware distributed LLM training.