A Cost-and-Time-Efficient Approach of Deploying Mixture of Expert Models

The emerging large language models (LLMs) such as ChatGPT and DeepSeek have shown great potential in almost all fields. However, the current centralized cloud deployment fashion makes it barely possible to jointly serve tremendous user requests due to shortcomings like insufficient resources and high response delays. In response to relieving the above shortcomings, one approach is to deploy the LLM model in edge networks, which shortens the responding delay and lowers the volume pressure. This work shows the very pioneering efforts in deploying the mixture of expert models (MoE) LLM over edge networks. To begin, we formally formulate the MoE deployment problem. Next, we propose a novel cost-closeness centrality (CCC) measure and design the novel CCC-based router and aggregator deployment (3C-RAD) algorithm. Through extensive simulations, we have found that the 3C-RAD algorithm can significantly improve the runtime efficiency while keeping the cost efficiency in deploying MoE compared to brutal force.