Soft Actor-Critic-Based Multi-User Multi-TTI MIMO Precoding in Multi-Modal Real-Time Broadband Communications

The next-generation wireless network is envisioned to support real-time broadband communication (RTBC) to provision services for immersive applications. Such applications (e.g., virtual reality, VR) usually need to simultaneously transmit multi-modal (e.g., visual, audio and haptic) data streams that have different traffic characteristics and transmission requirements, within multiple transmission time intervals (TTIs). In this paper, we formulate an optimization problem of multi-user multiple-input multiple-output (MIMO) precoding within multiple TTIs for multi-modal data transmission. As it is hard to find an optimal solution, we first resort to a novel soft actor-critic (SAC)-based learning approach. Specifically, a lightweight reinforcement learning architecture is employed to learn the adaptive priority weight of each user within multiple TTIs by taking into account its remaining multi-modal data amount and dynamic interaction state. The learned priority weights are then input to an iterative weighted minimum mean-square error (WMMSE) algorithm to adjust the precoder matrix and user transmission rates. With a scalable state design, the proposed algorithm can be tailored to different numbers of potential or active users. We also provide another practical solution to the formulated multi-TTI precoding problem, which transforms the problem into a single-TTI optimization problem by adding the quality-of-service (QoS) constraints into the traditional WMMSE problem and then solves it using the alternating direction method of multipliers (ADMM). Simulation results demonstrate the robustness and efficiency of the proposed algorithms, which show that the SAC-based precoding algorithm can achieve a 50.0% increment in system capacity compared to traditional WMMSE and a significant reduction in time complexity compared to the QoS-constrained WMMSE algorithm.