Deep learning for sequential decision-making problems in wireless systems

The recent advancements in deep learning, coupled with its integration into sequential decision-making frameworks such as dynamic programming, have transformed the approach to solving complex optimization problems in dynamic environments. In wireless systems, spanning both communication and sensing/localization, the need for intelligent and adaptive paradigms has grown increasingly critical. These systems operate in highly dynamic settings characterized by mobility, fluctuating channels, and varying performance demands, which render traditional myopic approaches inadequate. Deep learning enables the modeling of intricate dependencies, and its fusion with sequential decision-making frameworks allows for the approximation of optimal decision policies directly from data and system interactions. This combination facilitates adaptive responses to evolving conditions, making it essential for addressing the challenges and meeting the performance requirements of next-generation wireless networks. This dissertation develops deep learning-based sequential decision-making approaches for various settings and challenges in wireless sensing and communications. Specifically, it addresses the general problem of antenna/sensor selection for thin array design in wireless systems, a recurring issue tackled in prior work through supervised learning, convex optimization, or greedy methods. Here, the problem is reframed as a sequential decision-making task modeled as a deterministic Markov Decision Process. The Generative Flow Networks paradigm is adapted to learn an action-sampling policy, ensuring the probability of reaching each terminal state aligns with its reward. This approach outperforms greedy methods, convex optimization, and supervised learning across standard benchmarks. The second focus is mobile relay motion control. A deep reinforcement learning approach is proposed to optimize relay movement over time under spatiotemporally correlated channels, maximizing the cumulative SINR at the destination. The channel variability introduces high-frequency components into the optimal value function, addressed by integrating Fourier features into the neural network for improved value function estimation. The third setting involves designing Intelligent Reflective Surface (IRS) phase shift values for MISO communication systems under correlated channels. A deep reinforcement learning actor-critic method is developed, leveraging sufficient conditions on the critic's Neural Tangent Kernel to facilitate convergence under deep Q updates.