Two-Timescales Optimization of Content Placement and Delivery in Satellite-Terrestrial Edge Computing Networks

In this paper, we establish a two-timescale framework for the joint optimization for the content placement and content delivery problem in satellite-terrestrial edge computing networks (STECN). Our goal is to optimize content placement to improve network performance while ensuring diverse quality of service (QoS) for content delivery. We decouple the problem into two timescales to balance real-time responsiveness and long-term efficiency. Specifically, considering frequent content placement incurs huge traffic cost, we optimize the content placement in order to reduce resource expenses in large timescales. The optimization problem is formulated as an integer linear programming (ILP) problem to improve both traffic efficiency and cache resource utilization. We leverage a heuristic atom search optimization (ASO) approach to address the problem, which yields an optimal strategy with low computational complexity. In small timescales, we model content delivery as a Markov decision process (MDP) to minimize content delivery delays at small timescales while maintaining smooth network traffic. A deep reinforcement learning (DRL) framework is used for policy learning to dynamically adapt to varying network conditions. By considering the correlation between the small and large timescale optimization, we propose a hierarchical solution to jointly address both issues. Finally, extensive simulations confirm the effectiveness and superiority of the proposed scheme.