Quantum DRL for UAV-RIS-aided Maritime Communications with 6G Digital Twin Applications

This paper presents a digital twin (DT) framework for the sixth-generation (6G) maritime communication, targeting extreme ultra-reliable low-latency communication (xURLLC). In particular, we propose a DT model, where the physical system integrates unmanned aerial vehicles with reconfigurable intelligent surfaces and a high-altitude platform featuring mobile edge computing. We formulate a nonlinear programming problem to minimize total xURLLC user latency while satisfying energy and computational constraints. Quantum proximal policy optimization (Q-PPO) employed at the DT layer is adopted to cope with dynamic network conditions. The simulation results demonstrate superior performance for Q-PPO, which converges rapidly with minimal state space and requires fewer iterations per episode compared to classical PPO. Hybrid-Q-PPO, incorporating a parameterized quantum circuit within the policy network, also delivers notable performance enhancements than classical PPO. Moreover, Q-PPO achieves higher overall rewards than Hybrid-Q-PPO. Both Q-PPO and Hybrid-Q-PPO significantly reduce latency and improve resource utilization, highlighting the effectiveness of QDRL.