Enhancing Stability and Resource Efficiency in LLM Training for Edge-Assisted Mobile Systems

As mobile devices continue to drive advanced applications, edge computing has emerged as a crucial solution to overcome their inherent computational constraints, especially in deploying and training large language models (LLMs). Despite progress in edge computing, significant challenges remain in achieving efficient LLM training while addressing computational demands, energy consumption, and model stability. This paper presents an enhanced collaborative training framework that integrates mobile users with edge servers to optimize resource allocation. We extend the framework by incorporating model stability into the optimization objectives, mitigating performance instability often observed during distributed LLM fine-tuning. A multi-objective optimization problem is formulated to minimize energy consumption, delay, and instability, with a novel fractional programming technique and Iterative Rank Penalization (IRP) method proposed to improve the resource allocation and user-to-edge server associations. Compared to traditional methods like Semidefinite Relaxation, IRP achieves higher accuracy and computational efficiency. Extensive simulations demonstrate that our approach outperforms existing methods in reducing energy consumption and delay, and improving LLM stability across various mobile edge computing environments.