Hybrid Model-Based and Data-Driven DRL Optimization Approach: A Novel DRL Design to Accelerate Convergence

Deep reinforcement learning (DRL) has recently found comprehensive application in communication systems to address complex resource allocation problems. However, conventional DRL algorithms suffer from slow convergence, severely limiting their utility in dynamic communication environments. In this paper, we present a novel approach, namely hybrid model-based and data-driven DRL (HM-DRL) optimization, to tackle these challenges. In the HM-DRL, the original problem is split into inner and outer optimization tasks. The inner optimization is accomplished using low-complexity model-based methods, while the outer optimization employs DRL techniques. The HM-DRL scheme reduces the number of optimized parameters in outer optimization, thereby improving convergence. As a case study, we apply HM-DRL to optimize reflector configurations in a reconfigurable intelligent surface (RIS)-assisted orthogonal frequency division multiplexing (OFDM) system. By jointly optimizing the activation of antennas and associated reflectors at the RIS, our HM-DRL approach significantly accelerates learning speed, facilitating the discovery of near-optimal solutions.