Joint Information Freshness Optimization for Digital Twin Communication Infrastructure Through Hierarchical Deep Reinforcement Learning

In recent years, Digital Twin (DT) has emerged as a transformative paradigm for enabling the future of the Internet of Things. By mapping the real-time status of physical entities to their digital counterparts, DTs facilitate the creation of high-fidelity, interactive environments suitable for advanced simulation and deeper insight. One of the key challenges lies in achieving a sufficient level of convergence between Physical Twins (PTs) and their corresponding DTs. To tackle this challenge, we introduce a mobile edge computing environment that enables the coordination between PTs and DTs in Digital Twin Networks (DTNs) by offloading data transmission and processing tasks to the edge. A Hierarchical Deep Reinforcement Learning (HDRL) framework is proposed to improve DT-PT synchronization and enhance coordination effectiveness in optimizing information freshness across multiple action policies within the Digital Twin Communication Infrastructure (DTCI). Our approach is validated through a DTCI simulator, where comprehensive evaluations of age of information performance are conducted. Experimental results show that our HDRL-based solution significantly enhances the information freshness under constrained DTCI resources and across diverse environmental conditions.