A Cost-Optimization Approach of Placing Student Models in Edge Networks

The remarkable success of large language models (LLMs) like ChatGPT and Deepseek has spurred significant demand for their widespread placement. However, placing full-parameter LLMs in resource-constrained networks is often infeasible, prompting the adoption of knowledge distillation techniques that enable the compression of LLMs into smaller, domain-specific student models while preserving their performance efficiency. In edge networks, where clients are geographically distributed, multiple student models are placed to serve local requests, while a centralized teacher model handles more complex tasks. Here, a critical challenge arises in determining the optimal placement of these student models to minimize client access costs. Targeting this challenge, this work formulates the problem of student model placement in edge networks (SMP-EN) with the objective of cost optimization and demonstrates its NP-hardness. To solve SMP-EN, we propose an efficient algorithm called average-cost-first student model placement (ACF-SMP). Extensive simulation experiments demonstrate that our ACF-SMP algorithm significantly reduces the average expected client access cost compared to the benchmarks.